Calculation, representation, and maintanence of sharing information in mesh networks

ABSTRACT

A method for representing, in a network data structure, a minimum amount of protection bandwidth required to be reserved on each link in a mesh network, to restore service upon failure of another node or link in the network. The method includes (1) receiving a request for a new service in the network, wherein the new service is represented by a service data structure having an identification of each link and transit node in a primary path for the new service, (2) determining, using the network and service data structures, whether the new service requires additional protection bandwidth to be reserved on any link in the network, and (3) updating the network data structure if any additional protection bandwidth is determined to be needed. In one implementation the network and service data structures are vectors and the steps of determining and updating involve vector operations between these structures.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of the filing date of U.S.provisional application No. 60/459,163, filed on Mar. 31, 2003. Thesubject matter of this application is related to U.S. patent applicationSer. No. 10/639,728, filed on Aug. 12, 2003 as attorney docket no.Dziong 8-25-16-32, the teachings of which are incorporated herein byreference.

[0002] This application is one of a set of U.S. patent applicationsconsisting of Ser. No. 10/xxx,xxx filed as attorney docket no. Doshi56-5-21-17-33, Ser. No. 10/xxx,xxx filed as attorney docket no. Doshi57-6-22-18-34, Ser. No. 10/xxx,xxx filed as attorney docket no. Doshi55-7-23-15-35, Ser. No. 10/xxx,xxx filed as attorney docket no. Dziong9-1, and Ser. No. 10/xxx,xxx filed as attorney docket no. Doshi58-10-27-19-36, all of which were filed on the same date and theteachings of all of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

[0003] 1. Field of the Invention

[0004] The present invention relates to network communications and, morespecifically, to restoration path sharing.

[0005] 2. Description of the Related Art

[0006] Advances in wavelength-division multiplexing (WDM) relatedtechnologies have started to allow for routing and networking at theoptical layer (e.g., optical switch layer) of communications networks,providing a migration toward true optical-layer networking.Optical-layer networking associated with mesh-connected optical networkscreates the need for routing wavelength demands over the mesh opticalnetworks and an equivalent need for service recovery in the opticaldomain in the event of failures.

[0007] Traditionally, the optical-layer network provided for fastrecovery services, while data networks (e.g., Internet Protocol (IP)networks) were primarily targeted to achieve “best-effort” services.However, with the increasing use of data networks to carry time-criticaldata (e.g., voice-over-IP (VoIP) data), resiliency and fast recovery ofservice are becoming important features of data networks as well.Advances in traffic engineering of data networks, such as those coveredunder the umbrella of multiprotocol label switching (MPLS), haveprovided a framework for directing the flow of specific services withinIP networks.

[0008] For both optical-layer networking applications and datanetworking applications, distribution of topology information, and, inparticular, restoration, recovery, and sharing information, is importantto the efficient and timely recovery of services. However, many of therestoration and recovery schemes in the prior art fail to provide acompact representation of the relevant information, methods to utilizethe data in path calculations in consideration of multiple cost criteriain a way that benefit from sharing of protection link resources, andefficient, distributed mechanisms for making that data available whereand when it is needed.

SUMMARY OF THE INVENTION

[0009] Problems in the prior art are addressed in accordance withprinciples of the present invention by a shared mesh network (SMN) thatincludes components for compact representation and maintenance ofnetwork sharing information, dissemination of the sharing informationwithin the network, and improved service primary and restoration pathcalculation as a function of the sharing information.

[0010] In one embodiment, the SMN is an IP network, and sharingcomponents include extensions to the multiprotocol label switching(MPLS) traffic engineering specifications, specifically, open shortestpath first with traffic engineering extensions (OSPF-TE) and OSPF opaquelink-state advertisement option.

[0011] The SMN includes components that determine primary/restorationpath pairs in consideration of the grouping of certain links within thenetwork into shared-risk link groups (SRLGs). Links that belong to SRLGshaving a primary path link, are set to a link cost of infinity, whilethe remaining links that do not belong to such SRLGs are evaluated as totheir ability to support sharing of restoration bandwidth. If a linksupports sharing, its link cost is reduced by a factor R. This factor Rcan be a function of the sharing degree for the link as discussedpreviously. Ultimately, the adjusted link costs are used in aminimum-cost primary/restoration path calculation.

[0012] Components of the SMN distribute an accurate but optimized linkstate description in the form of an integer node-link or node vectors.The SMN features path-based recovery providing efficient sharing ofprotection capacity, while putting few requirements on intermediatenetwork elements. The SMN includes mechanisms for fast detection, fastfailure notification, signaling to enable protection and bandwidthsharing, and identification of locally cached vs. transmitted sharinginformation. Although not limited to optical applications, the presentinvention can be implemented in communication networks that transmitsignals between nodes using optical transmission technology.

[0013] In one possible implementation, the shared mesh network includes(a) two or more nodes, (b) two or more optical links interconnecting thenodes, (c) a network manager (either centralized or distributed) adaptedto control reservation of protection bandwidth for the links, and (d)components for calculating and updating sharing information. In thisimplementation, a first link in the network is part of two or moredifferent protection paths, where each protection path corresponds to adifferent primary path. The network manager makes use of the sharinginformation to determine how much protection bandwidth to reserve on thefirst link for the two or more protection paths in such a way that theprotection bandwidth reserved on the first link is shared between theprotection paths of the two or more primary paths. As such, the amountof protection bandwidth reserved on the first link can be less than thesum of the bandwidths of the two or more primary paths.

[0014] In one embodiment, the present invention is a method that may beimplemented by a network manager for a mesh network a plurality of nodesinterconnected by a plurality of links. According to the method,information associated with the mesh network is represented in a networkdata structure, wherein the network data structure comprises, for eachlink in the network and each node or other link in the network, arepresentation of a minimum amount of protection bandwidth required tobe reserved on said each link to restore service upon failure of saidnode or other link. A request is received for a new service in thenetwork, wherein the new service is represented by a service datastructure comprising an identification of each link and transit node ina primary path for the new service. Using the network and service datastructures, it is determined whether the new service requires additionalprotection bandwidth to be reserved on any link in the network, and thenetwork data structure is updated if any additional protection bandwidthis determined to be required for the new service.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Other aspects, features, and advantages of the present inventionwill become more fully apparent from the following detailed description,the appended claims, and the accompanying drawings in which:

[0016]FIG. 1 illustrates an exemplary shared mesh data network (SMDN)according to one embodiment of this invention.

[0017]FIG. 2 illustrates TABLE 1, which captures an example of theamount of restoration capacity required on each link for recovering fromevery possible single link or node failure in the SMDN of FIG. 1.

[0018]FIG. 3 illustrates TABLE 2, which captures an example of theamount of restoration capacity required on each link for every possiblesingle link or node failure in the SMDN of FIG. 1 after the restorationcapacity required to protect an additional primary path has been addedto already allocated restoration capacity of TABLE 1.

[0019]FIG. 4 depicts an exemplary restoration message processing methodfor source nodes and transit nodes in a mesh network.

[0020]FIG. 5 depicts an exemplary shared mesh data network supportingmultiple services via disjoint primary and restoration paths.

[0021]FIG. 6 depicts node-link aggregate vector and node-aggregatevector representations of sharing information.

[0022]FIG. 7 depicts node-link binary vector and node-binary vectorrepresentations of sharing information.

[0023]FIG. 8 depicts the exemplary shared mesh data network of FIG. 5with an additional service added.

[0024]FIG. 9 depicts the primary path node-link vector representationand the primary path node-vector representation of the new primary pathin the network of FIG. 8.

[0025]FIG. 10 illustrates a routine for calculating the link cost as afunction of sharing and link utilization.

[0026]FIG. 11 depicts an exemplary method for calculating a minimum-costprimary/restoration path pair for a new service in consideration of linkutilization, utilization threshold, administrative weight, and sharingdegree.

[0027]FIG. 12 illustrates one exemplary method for calculating primaryand restoration paths that takes into account bandwidth sharing forrestoration paths and considers the impact of shared-risk link groups(SRLGs).

[0028]FIG. 13 illustrates an exemplary approach to implementing step1206 of the method of FIG. 12.

[0029]FIG. 14 illustrates an exemplary connection setup procedureextension to support sharing in mesh networks.

[0030]FIG. 15 illustrates an exemplary procedure for determining aminimum-cost restoration path given a particular primary path.

[0031]FIG. 16 illustrates an exemplary procedure for calculating thecost of each restoration path Rj per step 1506 of FIG. 15.

[0032]FIG. 17 illustrates a portion of a mesh network.

[0033]FIG. 18 illustrates exemplary modules, representing one specificfunctional partitioning of components of an ingress node of the SMDN ofFIG. 1.

[0034]FIG. 19 illustrates exemplary modules included in intermediatenodes along a working path and a protection path of the SMDN of FIG. 1.

[0035]FIG. 20 illustrates exemplary modules, representing one specificfunctional partitioning of components of an egress node of the SMDN ofFIG. 1.

DETAILED DESCRIPTION

[0036] Reference herein to “one embodiment” or “an embodiment” meansthat a particular feature, structure, or characteristic described inconnection with the embodiment can be included in at least oneembodiment of the invention. The appearances of the phrase “in oneembodiment” in various places in the specification are not necessarilyall referring to the same embodiment, nor are separate or alternativeembodiments mutually exclusive of other embodiments.

[0037] Introduction

[0038] Significant research has been done into various restoration andprotection strategies for mesh networks at both the packet and opticallayers. Proposed architectures include centralized vs. distributed,pre-computed vs. computed on demand in real-time, and link-based vs.path-based. Characteristics that distinguish between these variousrestoration and protection strategies include recovery time, failurecoverage, and required recovery capacity. A survey of relevant researchas well as a specific distributed recovery strategy that providessub-second recovery times for carrier-scale mesh-based optical networksis addressed in Doshi, B. T., Dravida, S., Harshavardhana, P., Hauser,O., and Wang, Y., “Optical Network Design and Restoration,” Bell LabsTechnical Journal, January-March 1999 (herein “Doshi '99”), incorporatedherein by reference in its entirety.

[0039] So-called “shared mesh recovery” is a known concept in opticaltransport networks (e.g., SONET/SDH). In this scheme, when a failureoccurs, the network recovers service carried by the affected paths byusing recovery capacity that has been set aside on alternative routesfor this purpose. The recovery capacity can be shared over multiplefailure scenarios. This involves commensurate switching capability fromthe transport network elements (e.g., digital cross connects).

[0040] For both optical-layer networking applications and datanetworking applications, resiliency requires extra capacity in thenetwork sufficient to recover from any network failures such as fibercable cuts or network element failures. This extra capacity is commonlyknown as the restoration capacity. Normally the extra capacity is sizedfor recovering from only a single failure at a given time in thenetwork. This is due to the fact that the probability of occurrence ofmore than one failure during the time interval required to repair afailure is very insignificant, and therefore most of the time it issufficient to cover just a single network failure. This also providesopportunity to share restoration capacity between disjoint networkfailures. To understand sharing, realize that if two primary pathscarrying traffic are routed disjointly, then a failure affecting one ofthem will not affect the other. Now, if the corresponding protectionpaths are routed such that they share common links then the restorationcapacity on those links can be shared for restoring traffic between anyfailures along their primary routes. In general, sharing can be achievedby:

[0041] 1) realizing primary paths which do not share one or more linksalong their routes,

[0042] 2) realizing the links that are common along their restorationpaths, and

[0043] 3) sharing the restoration capacity on these common links acrossthe failure of any disjoint links along their primary routes.

[0044] Fast and guaranteed recovery requires that restoration paths needto be computed in advance of any failure and accordingly, restorationcapacity needs to be set aside. Efficient use of restoration capacitycan be achieved using sharing. Sharing can be achieved by realizing ateach link along the restoration path the amount of restoration capacityrequired to recover from each failure (link or node) in the network andthen comparing it with the primary path information for estimating anypossible sharing (i.e., the amount of already allocated restorationcapacity that can be used for protecting the primary path from a failurealong its route). In the case of a centralized implementation, all thisinformation is available at the server. Whereas, in case of adistributed implementation, each node keeps track of the sharing detailsfor each connected link in the downstream direction. This informationcan be built by passing the primary path information to each node alongthe protection path.

[0045] Sharing information enables computation of more efficient primaryand secondary paths. This suggests that when a node that is responsiblefor computing primary and secondary paths is aware of the sharinginformation (for a distributed implementation the information isdistributed across the nodes in the network), it will be able to computemore efficient paths. Thus, for a distributed implementation, it isdesirable to distribute the sharing information at each node across thenetwork. Usually the distribution of sharing information across thenetwork is a costly operation. Therefore, it is highly useful toabstract the most useful information and represent it in a compact formfor distribution. A link-state routing protocol (such as OSPF), whichalready disseminates topology information, can be used to distribute thesharing information.

[0046] Unless otherwise clear from the context, the process of“restoration” is synonymous in this specification with the process of“protection.” In general, such process involves various associatedrecovery actions following the detection of a failure, includingswitching to an alternative (i.e., restoration) path. Depending on theimplementation, such process may involve either pre-computation orreal-time computation of alternative paths. Pre-computation generallyexhibits shorter recovery times but may be less optimal than real-timecomputation, which benefits from the ability to take into account themost current state of the network demands and resources. Also dependingon the implementation, such process may involve either pre-allocation orreal-time allocation of bandwidth on alternative paths. Pre-allocationgenerally exhibits shorter recovery times but may be less desirable thanreal-time allocation as the former requires the use of the bandwidtheven in the absence of a fault for which it is reserved, while thelatter leaves the bandwidth available for opportunistic services that donot have a guaranteed quality of service. Also, depending on theimplementation, such process may involve either pre-establishment orreal-time establishment of alternative paths. Pre-establishmentgenerally exhibits shorter recovery times than real-time establishmentand is preferred if it is supported by the network elements.

[0047] The Shared Mesh Data Network

[0048]FIG. 1 illustrates an exemplary shared mesh data network (SMDN)100 according to one embodiment of this invention. A mesh network is anetwork topology (or a subset thereof) in which there are at least twonodes having two or more paths between them. A path is a set of one ormore links and zero, one, or more corresponding intermediate nodes thatinterconnect links. As shown, SMDN 100 includes six ingress/egress nodesN1 to N6 interconnected by seven bidirectional links L1 to L7. Thesenodes and links provide different paths between different pairs ofnodes. For example, links L2, L5, and L7 and nodes N2 and N3 provide apath between nodes N1 and N6. Similarly, links L2, L4, and L6 and nodesN2 and N5 provide another path between nodes N1 and N6.

[0049] The concepts of the present invention are discussed in thecontext of exemplary shared mesh data networks. However, without loss ofgenerality, they may be applied generally to any mesh network (e.g.,Sonet, ATM, frame relay, MPLS) that provides traffic-switching controlat the nodes incident to the links in the network. A mesh data networkis a network in which data is communicated asynchronously between nodesin packetized form and it is a specific case of the more general meshnetwork. In the present example, SMDN 100 supports the multiprotocollabel switching (MPLS) architecture standard per Rosen, E., et al.,“RFC3031—Multiprotocol Label Switching Architecture,” Network WorkingGroup, The Internet Society, January 2001, incorporated herein byreference in its entirety; however, as would be understood by oneskilled in the art, other data technologies that offer similar datatraffic engineering features (e.g., ATM and frame relay) could besubstituted for MPLS.

[0050] As illustrated, SMDN 100 has been provisioned with five primarylabel-switched paths (LSPs) LSP-1 to LSP-5. A label-switched path iseffectively a tunnel between two nodes that carries service trafficaccording to a predetermined route. For example, LSP-1 is a tunnel fromnode N3 to node N5 that follows the path N3-N2-N5. (LSP-1 couldalternatively be described by the links it traverses, i.e., L5-L4, or bythe nodes and links, i.e., N3-L5-N2-L4-N5; however, for this discussion,in cases where no ambiguity will arise from the node-only notation, thatnotation will be used.)

[0051] An LSP is typically considered to be unidirectional; however, forclarity of illustration, each LSP in FIG. 1 is represented by abidirectional path that could also represent the correspondingunidirectional LSP in the opposite direction. Further, each LSP may havean associated demand of any reasonable value (e.g., 2 Mbps, 10 Mbps, and100 Mbps); however, without loss of generality, for this example, eachof the LSPs in FIG. 1 is assumed to be of unit demand. Extensions topairs of unidirectional LSPs and variable demands would be understood toone skilled in the art.

[0052]FIG. 1 illustrates the five primary LSPs as follows: LSP-1 carriestraffic along path N3-N2-N5, LSP-2 carries traffic along path N4-N5-N6,LSP-3 carries traffic along path N2-N1-N4, LSP-4 carries traffic alongpath N2-N5, and LSP-5 carries traffic along path N2-N3-N6.

[0053] Disjoint Primary and Protection Paths

[0054]FIG. 1 also illustrates five exemplary protection paths LSP-6 toLSP-10 (also of unit bandwidth in this example) that have been reservedto use for protection of services given a failure of one of the primarypaths. LSP-6 (along path N3-N6-N5) protects LSP-1, LSP-7 (along pathN4-N1-N2-N3-N6) protects LSP-2, LSP-8 (along path N2-N5-N4) protectsLSP-3, LSP-9 (along path N2-N1-N4-N5) protects LSP-4, and LSP-10 (alongpath N2-N5-N6) protects LSP-5. For clarity, in FIG. 1, the tag for eachprotection LSP carries a parenthetical suffix indicating the primary LSPit protects. For example, LSP-6 (P1) indicates the protection role thatLSP-6 plays for primary LSP-1.

[0055] Note that the protection paths in the example of FIG. 1 arestrictly disjoint from the primary paths that they protect. In thiscontext, strictly disjoint means that the two paths in eachprimary/protection path pair share no common links or nodes other thantheir common ingress and egress nodes. Though the protection paths ofthis example are strictly disjoint. from the paths they protect, thisneed not be the case in all implementations of the present invention.There will be some cases in some variants of the invention where nostrictly disjoint path exists between two nodes to serve as a protectionpath. In other cases, other factors or constraints of the routingalgorithm may lead to selection of a protection path, for a givenprimary path, that is not strictly disjoint from the primary path (e.g.,the protection path may share some common links with the primary path).Thus, rather than being limited to strictly disjoint, the primary pathsin embodiments of this invention are at least partially disjoint fromtheir respective protection paths, where two partially disjoint pathshave at least two links that are different. With this definition, astrictly disjoint path is a particular type of partially disjoint path.

[0056] A number of different mechanisms exist in the art fordetermination of disjoint paths between nodes given various topology andtraffic information. Some exemplary algorithms are detailed in Doshi'99. Additionally, mechanisms exist for establishing primary andprotection LSPs, once they are determined, e.g., simple networkmanagement protocol (SNMP) per Network Working Group, “Introduction to aSimple Network Management Protocol (SNMP) version 3,” RFC2570, April1999 (herein “RFC2570”) and RSVP-TE as covered by Awduche, D., Berger,L., Gan, D., et al., “RSVP-TE: Extensions to RSVP for LSP Tunnels,” RFC3209, December 2001 (herein “RFC 3209”), each of which is incorporatedherein by reference in its entirety.

[0057] Link Capacity

[0058] In the embodiment of the invention illustrated by the example ofFIG. 1, to provide for full recovery of services given any single-linkfailure, the capacity of a link must be large enough to accommodate theprimary services that it supports as well as the services it protects.For example, L5 is one link in primary service paths LSP-1 and LSP-5. Itis said to “support” both of those paths. Assuming that LSP-1 and LSP-5each carry one unit of demand bandwidth, L5 will need at least two unitsof allocated capacity to support those two paths. L5 is also part of theprotection path (LSP-7) for service path LSP-2 (requiring an additionalone unit of capacity). Thus, in this example, the capacity of L5 must beat least three units of bandwidth: two units of allocated bandwidth andone unit of reserved bandwidth.

[0059] Reservation vs. Allocation

[0060] In one embodiment of the present invention, the bandwidthassociated with the protection path (LSP-7) for service path LSP-2 isnot allocated in advance of a failure, but is, instead, only reserved.If the bandwidth were allocated in advance of a failure, this wouldcorrespond, in the parlance of the field of protection and restorationfor optical transport networks, to a 1:1 protection scheme. If thebandwidth were not only allocated, but additionally if a copy of theservice path's data were to be duplicated to the protection path; thiswould correspond to a 1+1 protection scheme. However, in preferredembodiments of this invention, the protection bandwidth is not allocateduntil after the detection of a failure. Thus, in such implementations,this unallocated bandwidth might be used for “opportunistic” data, e.g.,data that has a lower guaranteed quality of service (QoS) than theprotected traffic. Further, because the protection bandwidth is onlyreserved and not allocated, “sharing” can be supported, as definedbelow.

[0061] Sharing and Single Failure Coverage

[0062] Another characteristic of certain networks of this invention istermed “sharing.” This refers to the facility to share reservedprotection capacity on a link between more than one LSP. As describedpreviously, the primary paths provisioned on the SMDN are assumed to beat least partially disjoint from their respective protection paths. Thismeans that, for at least one link in each primary path, there is adetour segment in the corresponding protection path that has beendesignated to carry reserve capacity to protect the primary path.Recovery from a failure of a link along a primary path that has acorresponding detour segment in the protection path is achieved byswitching the affected traffic at the ingress node from the failedprimary path to the corresponding protection path. For a successfulrecovery, at the time of protection switching, enough capacity should beavailable on all links along the protection path. One way of achievingthis is by allocating dedicated bandwidth along the protection path ofeach LSP. This will result in excessive use of protection bandwidth. Amore efficient scheme involves sharing the protection capacity of a linkbetween LSPs that are not generally affected by the same single node orlink failure.

[0063] For example, consider the protection capacity that is set aside(i.e., reserved) on a link along the protection path to recover from afailure along a first primary path. Next assume that this link is alsodesignated as part of the protection path of a second primary path thatis disjoint from the first primary path (in this case, the secondprimary path would not be affected by a failure along the first primarypath). Then, given the assumption that no more than one link will failat a given point in time (a reasonable assumption given themean-time-between-failure (MTBF) statistics of state-of-the-artnetworks), the two primary paths may “share” the reserved protectionbandwidth of the protection link.

[0064] An example should help clarify the concept. Consider LSP-3 ofSMDN 100 as the first primary path. It is protected by LSP-8, whichincludes L4 (i.e., L4 is a protection link for LSP-3). L4 also serves aspart of the protection path (LSP-10) of LSP-5, where LSP-5 does notshare any links with LSP-3 (i.e., LSP-3 and LSP-5 are disjoint). Thus,only one unit of capacity needs to be reserved on L4 to protect againsta failure affecting either LSP-3 or LSP-5. LSP-3 and LSP-5 areconsidered to “share” this protection capacity on L4. The extension ofthis concept to the full SMDN is considered to be a generalized sharedmesh technique. Node failures represent another basis for protectionbandwidth reservation and may similarly benefit from sharing.

[0065] Worst-Case Link Protection Capacity

[0066] The amount of bandwidth reserved on a given link for protectionpurposes is chosen to accommodate the worst-case traffic demand thatwould be placed upon that link given the failure of any other one linkor any one node in the network. As another example of sharing and tohelp clarify this concept, consider link L6 of SMDN 100.

[0067] Link L6 supports both LSP-6 and LSP-10, the protection paths forLSP-1 and LSP-5, respectively. LSP-1 includes intermediate node N2 andlinks L4 and L5, while LSP-5 includes intermediate node N3 and links L5and L7. Since N2 is a terminal node of LSP-5, service protection forLSP-5 is not available in the event of a failure of N2. As such, onlyone unit of reserve capacity on link L6 is needed to protect LSP-1 froma failure of N2. Similarly, since N3 is a terminal node of LSP-1,service protection for LSP-1 is not available in the event of a failureof N3. As such, only one unit of reserve capacity on link L6 is neededto protect LSP-5 from a failure of N3. A failure of link L4 would resultin a failure of LSP-1, creating a need for one unit of reserve capacityon link L6. Similarly, a failure of link L7 would result in a failure ofLSP-5, also creating a need for one unit of reserve capacity on link L6.Since L4 and L7 are independent links, a failure of only one or theother (but not both) is all that needs to be considered. Thus, thereserve capacity required on L6 could still be capped at one unit.

[0068] However, if a failure on L5 is considered, both LSP-1 and LSP-5are affected. Since this one failure could result in a concurrent demandof two, a “worst case” of two units of bandwidth would have to bereserved on L6 to ensure maximal recovery of the network under theassumption of a maximum of one link or node failure at time. In general,in the calculation of the reserve capacity required for each link ofSMDN 100, the effect of the failure of each node and each of the otherlinks in the network is independently considered and the worst-casecapacity is reserved.

[0069] Tabulation

[0070] The concepts of the previous sections are quantified by TABLE 1of FIG. 2. TABLE 1 shows the amount of protection capacity required oneach link for recovering from every possible single link or node failurein SMDN 100 of FIG. 1. Each row of TABLE 1 corresponds to a differentlink of SMDN 100. For each row (corresponding to a particular link),each column entry identifies the protection capacity for that link inorder to protect service in the event of a failure of the link or nodeassociated with that column in the table. Using a table such as TABLE 1,the sharing of protection capacity between different failures can berealized.

[0071] To better understand TABLE 1, consider the first row, which isassociated with link L1. The entry of “1” in the column labeled L3 forthe first row indicates that there is one unit of traffic on link L3(due to LSP-2) that would employ link L1 on its recovery route if linkL3 gets impacted by a failure. Similarly, the entry of “1” in the columnlabeled N5 addresses the case of a failure of node N5 and its impact onlink L1. The last column titled Max is the maximum value of all entriesin that row. It is the amount of protection bandwidth that needs to bereserved on that link for the worst-case single failure in the network.This value for link L6, for example, is 2 units to cover the case of afailure of link L5 as discussed in the previous section (and asreflected by the entry of “2” in row L6, column L5 of TABLE 1). Theother entries of TABLE 1 are determined similarly.

[0072] Completion of the information in TABLE 1 and the calculation ofthe Max value enable the determination of how much protection capacityto reserve on each link in the SMDN to realize full recovery of servicesin the SMDN in the event of a single link or node failure.

[0073] Distributed vs. Centralized Sharing Database

[0074] The information in TABLE 1 can be either maintained in acentralized fashion at a server or distributed to the nodes in the SMDN.In case of a centralized architecture, TABLE 1 might reside on acentralized server and be updated after the provisioning of each newprimary and protection LSP. Signaling could be used to notify nodes ofany change in the reservation bandwidth on any of their connected links.

[0075] In a preferred distributed implementation of SMDN 100, portionsof the information from TABLE 1 are distributed to nodes throughout thenetwork. In one implementation, each node calculates the amount ofreserved protection capacity on each of its incident links andbroadcasts that information to the other nodes in the network. Forexample, node N5 in SMDN 100 keeps track of the protection capacityreserved on links L3, L4, and L6. This corresponds to the informationfrom TABLE 1 in the rows labeled with those link designations (i.e., L3,L4, and L6). Likewise, node N4 keeps track of information relevant tolinks L1 and L3 (corresponding to the rows labeled L1 and L3 in TABLE1). Note that nodes N4 and N5 both keep track of information relevant tolink L3. A secondary mechanism (e.g., periodic refresh or localizedflooding) is used to keep the (now distributed) information of TABLE 1current and up-to-date at the other nodes in the network. Each node canalso keep track of the full network information (e.g., the completeinformation in TABLE 1) for use in calculation of primary andrestoration paths as needed to service new bandwidth requests.

[0076] Calculation and Distribution of Sharing Database

[0077] In steady-state operation of a shared mesh data network, it isassumed that once a new demand (i.e., service request) is received at anode, a route computation routine is invoked that calculates both aprimary path (also known as “working path”) as well as a protection path(also known as “restoration path”) for the demand. Signaling (e.g.,SNMP, PNNI, or RSVP-TE signaling) is used to establish the primary pathfor the service and to reserve bandwidth for the protection path. Thesignaling along the protection path carries the information of theprimary path in terms of its demand bandwidth as well as nodes and linksit contains. For each of the links incident to a node in the protectionpath, the node updates its local reserved bandwidth database(corresponding to its portion of TABLE 1). This update involvesincrementing, by the requested demand bandwidth, the value in each entryof each relevant row that corresponds to a link or node in the primarypath (i.e., those links or nodes that, upon failure, would cause adisruption in a service along the primary path and thereby levy abandwidth demand on the links in the corresponding protection path). Theupdate also involves calculating the maximum value of each updated row.Note that a link connecting two nodes can be a downstream link for eachnode in different directions. It is assumed that for a particularprotection path, the upstream node to the link computes and updates thesharing information (for an optical network, this may also includeselection of the time slots, wavelength, and ports) and then passes thisinformation to the downstream node so that both nodes connected to linkhave the same view of resources and sharing for the link. This scheme isspecifically useful in cases where primary and protection paths arebi-directional.

[0078] As an example, consider the arrival of a request for shared meshprotection service between nodes N4 and N2 of SMDN 100. Assume that, atthe arrival of the request, the network was in a state captured by FIG.1 and TABLE 1 of FIG. 2. Further, assume that LSP-11 (i.e.,N4-L3-N5-L4-N2) and LSP-12 (i.e., N4-L1-N1-L2-N2) are the primary aprotection paths, respectively, determined to service this request. Asbefore, signaling along the primary path is used to establish orprovision the primary LSP. Also, as before, signaling along theprotection path is used to carry the route information as well as thebandwidth demand (assumed in this example to be one unit) of this newservice to all the nodes (i.e., N4, N1, and N2) in protection pathLSP-12 for the new primary path LSP-11. As a result, rows L1 and L2 ofTABLE 1 will be updated (centrally or, in the case of a distributedarchitecture, at nodes N4 (just row L1), N1 (both rows L1 and L2), andN2 (just row L2)). The update result is captured in TABLE 2 of FIG. 3.Note that an additional one unit of bandwidth is now needed on bothlinks L1 and L2 to guarantee the recovery of the new connection request(LSP-11) from a failure along its working route (i.e., N4-L3-N5-L4-N2).

[0079] Note that the availability of the complete sharing-informationtable (e.g., TABLE 1 or TABLE 2) at each node allows for more efficientcomputation of primary and protection paths; however, this is not arequirement for achieving sharing. In the case of a centralizedarchitecture, this capability comes for free, since the completesharing-information table resides on a centralized server and may bemade available to each node through a centralized command signalingstructure (e.g., SNMP). In the case of a distributed architecture,similar capability can be achieved at the cost of periodicallyexchanging the information of the complete sharing-information tablebetween nodes. Thus, a centralized approach provides savings in terms ofthis inter-nodal exchange of information, while a distributed approachprovides additional robustness (e.g., the elimination of a single pointof failure) and scalability.

[0080] Connection Routing and Service Set-Up Process

[0081] In one or more embodiments of the present invention, a failurealong a primary path for a service is typically detected at thedestination node of the service and signaled to the source node for theservice. In response to this failure signal, the source node thenswitches traffic for the service over to a pre-established protectionpath that is disjoint from the primary path. The bandwidth to supportthe service is typically reserved (but not allocated) on the protectionpath at the time of the primary service setup. Note that the capacity onthe restoration path, including the bandwidth reserved to support theprimary path just discussed, is potentially shared among multiple otherservices that do not have common elements in their primary paths. Notethat in case of shared restoration, the source first establishes apre-computed restoration path and then switches the traffic to it. Forachieving sharing in optical networks, one would use shared restorationas opposed to shared protection because an established optical pathphysically occupies resources such as time slots, wavelengths and thusdoes not allow their sharing until they are released by deleting theoptical path.

[0082] In one embodiment of the present invention, a network manager isresponsible for the determination of primary and restoration paths forservices in the network such that sharing is increased and hence networkcost (in terms of bandwidth or other cost elements) is decreased,network revenue is increased, and/or sufficient capacity is reserved onthe restoration path to provide for reasonable robustness of the networkto failures.

[0083] In a different embodiment, distributed calculation of primary andrestoration paths is performed in the network elements, as opposed tocentralized calculation, since it provides better scalability andreliability. Whenever a connection request arrives at a source node, adistributed element of the network manager has the task of calculatingcost-efficient primary and restoration paths. To do this, the portion ofthe network manager associated with the source node needs informationabout the link states in the network. Nodes maintain local link-stateinformation about the links incident to them. This information, alongwith sharing information (which can be classified as additionallink-state information), is advertised, using a data network, to othernodes in the network. Hence, all nodes in the network, and ultimatelythe network manager (which may be a stand-alone, centralized entity, adistributed entity, or intrinsically a part of one or more of the nodesin a network), will eventually have the information on the state andsharing characteristics of all the links in the network that is neededto make the primary and restoration path calculations for this newconnection. (Note that link-state information describing topology andbandwidth availability provides sufficient information to compute alegitimate pair of primary and restoration paths. The sharinginformation, however, additionally allows for the computation of a moreefficient primary/restoration path pair). Once the paths are calculated,service is set up on the primary path, and the required bandwidth forthe service's restoration path is reserved on the restoration path.

[0084] The process of reserving bandwidth on the restoration path issimilar to the process of connection setup on the primary path. Inparticular, a reservation message is sent to along the restoration pathto each of the transit (i.e., intermediate) nodes along the path. Thisreservation message is analogous to the connection setup message that issent along the primary path. One difference between the two messages isthat the reservation message contains not only restoration pathstructure information, but additionally information about: the primarypath structure, while the connection setup message contains structureinformation about only the primary path. Using this additionalinformation, transit nodes along the restoration path (or a regionalmanager that is responsible for one or more of these transit nodes) canassess whether sharing of link restoration bandwidth is possible.

[0085] In particular, each transit node (or its regional manager)executes a process similar to that illustrated by the flow diagram ofFIG. 4 for its incident links that are part of the restoration path.Note that each node along the restoration path (including the sourcenode) verifies, for the link connected to it in the downstreamdirection, whether it is possible to use already allocated restorationcapacity for protecting the primary path and, if not, it allocates anyadditional capacity that is required on the link as long as thatcapacity is available. It also updates the sharing information for thelink accordingly. Therefore the processing in FIG. 4 applies to all thetransit nodes along the restoration path except for the destinationnode.

[0086] Namely, in step 400, the node receives and parses the restorationmessage and determines the bandwidth requirements that would be leviedon the link (that is connected to it in the downstream direction alongthe restoration route) if there were to be a failure in one of the linksthat make up the primary path. In step 402, the node compares thisbandwidth with the existing bandwidth requirements of primary links forwhich it is currently providing restoration bandwidth and determines ifthis new restoration path bandwidth can be shared with other restorationpaths that are already reserved on the link.

[0087] If the bandwidth can be shared, then, in step 404, the nodeaccepts the restoration path for the link. Next, in step 406, the nodeupdates the sharing information associated with the link, and, exits theprocess in step 408.

[0088] If the bandwidth for the new restoration path cannot be shared,then, in step 410, the node determines whether the new restoration pathbandwidth requirements can be supported by the existing capacity of thelink. If it can, then, in step 412, the node reserves the necessaryadditional bandwidth on the link and the process proceeds to step 404 asdescribed previously. If, in step 410, it is determined that theadditional bandwidth required by the new restoration path would requirean aggregate reservation of bandwidth on the link that exceeds thecapacity of the link, then, in step 414, the node rejects therestoration path for the link, and, in step 408, the process exits.

[0089] Note that the above process can be performed in parallel on eachof the transit nodes in the network.

[0090] For more optimal computation of primary and restoration paths ina distributed implementation, it is desirable (though not necessary) todistribute the changes in the links-state or sharing information to theother nodes in the network. However, disseminating this information forany possible change is quite expensive in terms of bandwidth andprocessing required. Therefore, typically, various methods are employed(e.g., based on thresholds) to limit the frequency of dissemination.

[0091] When sharing information is also being disseminated to othernodes along with the traditional link-state information, then, inaddition to limiting the dissemination frequency, it is highly desirableto also address the size of the information itself. A reduction in thesize of the sharing information can be achieved by using a scheme torepresent the information in a compact form, by only disseminating themost critical information, or by using a combination of both.

[0092] Link-State Descriptions

[0093] In one distributed implementation of the present invention, wherethe computation of primary and restoration paths as well as initiationof their setup is distributed across the network, it is typical for onlythose nodes connected to a link to have complete visibility into thestate of bandwidth allocations and bandwidth sharing on the link. Someof this information (e.g., available and currently allocated bandwidthon the link) should be distributed to other nodes in the network so theycan compute primary and restoration paths. (Note that without thisinformation the nodes may end up computing paths through congestedsegments of the network, thereby causing unnecessary crankbacks. Thisprocess can go in cycles and can significantly degrade networkperformance. Therefore, typically it is assumed that some minimalbandwidth information is disseminated to allow computation of a paththrough the network).

[0094] Link-state structure of traditional link-state protocols (OSPF,ISIS) has already been extended to include this necessary bandwidthinformation. The extensions are referred as the traffic engineeringextensions (OSPF-TE, ISIS-TE).

[0095] In addition to bandwidth information, it is also desirable toexchange bandwidth-sharing information between nodes. Bandwidth sharinginformation allows computation of more optimal paths (e.g., that improvesharing) that can considerably reduce the required amount of restorationbandwidth, and thus the overall cost of the network. However, beforeemploying any scheme (link-state routing, signaling, etc.) to distributethe sharing information, it is highly desirable to define compactrepresentations of the data so dissemination of the data does not becomea significant burden on the routing (OSPF-TE, ISIS-TE) and signaling(RSVP-TE, CR-LDP) protocols.

[0096] Conceptually, bandwidth-sharing information that gives the stateof sharing at each link can thus be classified as link-stateinformation. A number of different compact representations can be usedto represent the sharing information in the link-state structure. Thesedifferent compact representations may provide less than the full amountof information available regarding the sharing. However, beneficially,these representations take fewer bits to communicate the informationused by the manager to update link-state information globally and tocalculate primary and restoration paths associated with new servicerequests. The link-state descriptions are best illustrated in thecontext of a link that is part of the restoration path of a number ofprimary paths, such as link L5 of FIG. 5.

[0097]FIG. 5 illustrates exemplary network 500, which includessub-network 502, sub-network cloud 504, and exemplary restoration-pathlink L5. Sub-network 502 includes nodes NA, NB, NX, NY, and NZ and linksL1, L2, L3, and L4. Link L5 is bounded by nodes NQ and NR and isconnected to sub-network 502 via sub-network cloud 504. The actual linksconnecting link L5 to sub-network 502 are not pertinent to thisdiscussion and are thus not shown explicitly but instead are abstractedby cloud 504.

[0098] In the present example, network 500 currently supports fourdifferent services on four primary paths. The first service is betweennodes NX and NZ and is carried by a primary path along L1-L2-L4. Thesecond service is between nodes NA and NY and is carried by a primarypath along L2-L3. The third and forth services are between NB and NY andthey are carried by two different primary paths along L3.

[0099] The four services are protected by restoration paths R1, R2, R3,and R4. Note that each of these restoration paths includes link L5. Inthe following sections, various representations of the sharinginformation will be described using L5 as an example.

[0100] Aggregate Node-Link Vector

[0101] A first representation of the sharing information for L5 is theaggregate node-link vector representation. This representation capturesminimum sufficient sharing information that is required for a link tocorrectly compute the amount of restoration capacity required on it toprovide guaranteed restoration to an additional primary path whilerealizing sharing, and to correctly deduce any required release ofrestoration capacity on deletion of a primary path. Note that thisrepresentation is very closely related to the row representation asillustrated in TABLE 1 and TABLE 2 of FIGS. 2 and 3, respectively.

[0102] In this representation, a vector is formed such that each elementof the vector corresponds to a component in the network (e.g., a node orlink) whose failure might contribute to the bandwidth reservationrequirements of link L5. For the exemplary network of FIG. 5 and thefour exemplary primary path services, the elements include NA, NB, L1,L2, L3, and L4. (Note that, for the present example, nodes NX, NY, andNZ are not considered as elements of the aggregate node-link vector,because they are terminal nodes (i.e., source or destination nodes) andnot intermediate nodes for the four services protected by link L5.) Thevalue in each position of the vector represents the aggregate bandwidththat would be required on link L5 in the event of a single failure inthe network of the network element that corresponds to that vectorposition. FIG. 6 illustrates aggregate node-link vector V_(nla) for linkL5 associated with the exemplary network 500 of FIG. 5.

[0103] For this example, it is assumed that the bandwidth demand of eachservice is of unit value, however, extension to the more general case ofnon-unit bandwidth demands would be understood to one skilled in theart. As illustrated, the value of the vector in the positioncorresponding to node NA is 1. This indicates that, for a failure ofnode NA, an aggregate bandwidth demand of 1 (associated with protectionpath R1) will be placed on link L5 corresponding to one (unit bandwidth)service. Similarly, the 2 in the position of V_(nla) corresponding tonode NB indicates that for a failure of node NB, link L5 will berequired to support two aggregate (unit bandwidth) services paths (i.e.,associated with protection paths R1 and R2). Similarly, potentialfailures of links L1, L2, L3, and L4 correspond to aggregate bandwidthreservation values in V_(nla) of 1, 2, 3, and 1, corresponding torestoration paths R1, R1+R2, R2+R3+R4, and R1, respectively. Note thatthe bandwidth that is reserved on link L5 is the maximum of thebandwidth requirements associated with any single network componentfailure. In the example network of FIG. 5, the reservation bandwidth isthus max {1, 2, 1, 2, 3,}=3.

[0104] As discussed earlier, to provide sharing, each node in thenetwork keeps track of the aggregate node-link vector for all directlyconnected links. To share this information across the network (usefulfor computing more optimal primary and restoration paths), in oneimplementation, each node disseminates aggregate link-node vectors forits connected links. For a large network, this may turn out to beexpensive because of the sheer size of each aggregate node-link vector.Other implementations involve sending a more compact representation ofthe sharing information. This compact representation, which summarizesthe aggregate node-link vector information, is still useful in computingmore efficient primary and restoration paths. Some of the compactrepresentations are described in the following section.

[0105] Node-Aggregate Vector

[0106] An alternative representation of the link-state sharinginformation is referred to as a node-aggregate vector, designatedV_(na). FIG. 6 also illustrates the node-aggregate vector V_(na) forlink L5 of exemplary network 500 of FIG. 5 as well as its derivationfrom the aggregate node-link vector V_(nla). As illustrated, eachelement of V_(na) corresponds to a node in the network (although not allnetwork nodes need to be included in V_(na)) and is a function of thenode itself, specifically, the reservation values associated with alllinks incident to the node. The value 2 corresponding to node NA, forexample, is derived from V_(nla) by taking the maximum of thereservation values for node NA and links L1 and L2, which, as can beseen from FIG. 5, are the links incident to node NA. Similarly, thevalue 3 corresponding to node NB is derived from V_(nla) by taking themaximum of the reservation values for node NB and its incident links L2,L3, and L4.

[0107] Binary Node-Link Vector

[0108] Another alternative representation of the link-state sharinginformation is referred to as a binary node-link vector, designatedV_(nlb). FIG. 7 illustrates the binary node-link vector V_(nlb) for linkL5 of the exemplary network of FIG. 5. The binary node-link vectorV_(nlb) is created by a transformation of the aggregate node-link vectorV_(nla). Namely:

V _(nlb)(j)=1; if V _(nla)(j)=reservation bandwidth for the link

V _(nlb)(j)=0; otherwise.

[0109] In other words, the binary node-link vector V_(nlb) indicateswhich network element failure will cause activation of all the bandwidththat was reserved for restoration paths on the link. So, for example, asillustrated in FIG. 7, all elements of vector V_(nlb) are zero, exceptfor the element corresponding to link L3, since only the failure of linkL3 will require all of the bandwidth (i.e., 3 units) reserved on linkL5.

[0110] Node-Binary Vector

[0111] Yet another alternative representation of the link-state sharinginformation is referred to as a node-binary vector, designated V_(nb).FIG. 7 also illustrates the node-binary vector V_(nb) for link L5 of theexemplary network of FIG. 5 as well as one method for its derivationfrom the binary node-link vector V_(nlb). As illustrated, each elementof V_(nb) corresponds to a node in the network (although not all networknodes need to be included in V_(nb)) and is a function of the nodeitself, specifically, reservation values associated with all linksincident to the node. The value 0 corresponding to node NA, for example,is derived from V_(nlb) by taking the maximum (or the binary OR) of thereservation values for node NA and its incident links L1 and L2.Similarly, the value 1 corresponding to node NB can be derived fromV_(nlb) by taking the maximum (or the binary OR) of the reservationvalues for node NB and its incident links L2, L3, and L4. Alternatively,the binary node vector V_(nb) can be derived from the node-aggregatevector V_(na) as follows:

V _(nb)(j)=1; if V _(na)(j)=reservation bandwidth for the link

V _(nb)(j)=0; otherwise.

[0112] Note that V_(nb) can also be derived directly from V_(nla).

[0113] Other alternative representations of the link-state sharinginformation, including inverted binary, real-number valued, and deltarepresentations relative to other elements of the vector or a constant,as well as compressed (e.g., entropy, Huffman, or run-length encoded)versions can be used as would be understood to one skilled in the art.

[0114] Note that the previous descriptions of vector representations ofsharing information assumed a homogeneous network where all servicerequests are of unit bandwidth. Extension of these concepts to casesinvolving non-unit, real-valued bandwidths of new and existing serviceswould be understood to one skilled in the art.

[0115] Sharing Determination and Updating Sharing Information

[0116] To understand the operation and use of the sharing informationvectors, consider the addition of a new service to exemplary network 500of FIG. 5 as illustrated in FIG. 8. A primary path node-link vectorV_(pnl) for the new service 802 of FIG. 8 is illustrated in FIG. 9. Avector element of a primary path node-link vector V_(pnl) is assignedthe bandwidth of the service demand if the element position correspondsto a network component that is in the primary path and a value of zerootherwise. So, the primary path node-link vector V_(pnl) for new service802 includes ones (because, in this example, we are assuming unitdemands) in positions corresponding to node NB and links L2 and L4 andzeros in the other positions.

[0117] Assuming the restoration path for the new service path traverseslink L5, it is of interest to determine whether this new restorationpath can share the bandwidth of this target link L5 with its presentlysupported restoration paths.

[0118] Aggregate Node-Link

[0119] One way to make this determination is to add the primary pathnode-link vector V_(pnl) associated with the new service path to theaggregate node-link vector V_(nla) that was associated with the previousaggregate reservation on target link L5. This addition is done on anelement-by-element basis to form a sum vector V_(snl). The maximum valueof this resulting sum vector is then compared to the current reservationbandwidth of link L5. If the maximum value is equal to the currentrestoration bandwidth, then the restoration path for the new service canshare the bandwidth reserved for the restoration paths of the existingservices. Otherwise, supporting the new restoration path will requirereservation of an additional bandwidth unit. In the case of theexemplary new service 802 of FIG. 8, the sum vector V_(snl) resultingfrom the vector addition of the primary path node-link vector V_(pnl) ofFIG. 9 and the aggregate node-link vector V_(nla) of FIG. 6 is{1,3,1,3,3,1}, where the maximum value is three, which is equal to thecurrent reservation bandwidth on link L5. Thus, the restoration path forthis new primary path may be added to link L5 without the reservation ofadditional bandwidth on link L5.

[0120] The determination of whether link L5 can be shared corresponds tostep 402 of the method of FIG. 4, which is performed by a node incidentto link L5 after receiving a restoration path request message.Continuing this example, following this determination of the ability toshare in step 402, the restoration path bandwidth is accepted (step 404)and the sharing information associated with link L5 is updated (step406). This updating is done by setting the updated aggregate node-linkvector V_(nla) ^(u) equal to the sum vector V_(snl) or, in other wordsV_(nla) ^(u)=V_(nla)+V_(pnl). A subsequent determination of sharingability for link L5 would then be performed using V_(nla) ^(u).

[0121] To calculate, in an optimal way, the primary and restorationpaths at the source node, one could advertise sharing information in thenetwork using the aggregate node-link vector form or one of its compactrepresentations so that each node would have information about whichlinks can share the restoration bandwidth. Note the compactrepresentations only provide a summary of the sharing information.

[0122] Computation of Routes Based on Multiple Cost Criteria

[0123] The link state information (bandwidth allocation) and sharinginformation as outlined previously can be used to compute cost-efficientconnection routes within a network. For path computation, link state canbe reduced to an equivalent link cost and then route-computationalgorithms can be considered that minimize the total path cost—where thepath cost is considered to be the sum of the link costs. The link-costfunction should be carefully designed. Also, depending on thepossibility of sharing bandwidth on the restoration path, link costswill be different for the sharing case compared to the no-sharing case.(Note in case of sharing case, sharing information can be used tocompute more cost-efficient paths through the network. This can beachieved by incorporating sharing information in the link-cost metric).The two cases are described below.

[0124] No-Sharing Case

[0125] Link-cost calculation is based on the administrative weight (AW)of the link, the link capacity (LC), and link's available capacity (AC).Under light and medium link utilization (LU), where LU is less than aspecified utilization threshold (UT) (i.e., LU≦UT), link cost whensharing is not considered w^(NS) is set equal to the AW, i.e.,w^(NS)=AW. Hence, under light load conditions, the link-calculationalgorithm will assign links to services according to the preferences(i.e., administrative weights) assigned by the network operator. Whenthe link load is high (LU>UT), however, the link weights are preferablycalculated with the objective of load balancing and revenuemaximization. The generic formula for the link cost w^(NS) in this highlink utilization region is based on the inverse of available capacity:$\begin{matrix}{w^{NS} = \frac{{AW} \cdot {MWC}}{A\quad C^{f}}} & (1)\end{matrix}$

[0126] where MWC is maximum weight coefficient (i.e., an additionalscaling factor introduced to provide continuity between the two regionsof light and heavy loads) and f is an exponentiation factor (nominallyset to 0.4) used to modify the available capacity variable AC.Motivation, a detailed model, and additional numerical support for thisapproach of weight calculation based on the inverse of availablecapacity are described in Dziong, Z., “ATM Network Resource Management,”McGraw-Hill, 1997, (herein “Dziong '97”) incorporated herein byreference in its entirety.

[0127] Sharing Case

[0128] When sharing information is available, it can be used to computemore cost-efficient (more optimal) primary and restoration paths. Forexample, an algorithm can be designed to compute, for each possibleprimary path, the lowest-cost restoration path by utilizing the sharinginformation. Then, from the set of all primary and restoration paths,the pair that requires the least amount of additional capacity can bechosen. This path computation algorithm using the sharing informationcan give considerably better paths than an algorithm using no sharinginformation.

[0129] Sharing information can be used in finding the least-costrestoration path for a given primary path. It involves adjustment of thelink cost (lowering of it) based on the amount of sharing that ispossible on a link along the restoration path of a particular primarypath. This can be achieved by computing the sharing degree of each linkin the network given the primary path. (Note that only links that aredisjoint to the primary path need to be considered).

[0130] The sharing degree is defined as the maximum number of additional(unit bandwidth) primary services (along the proposed primary path) thatcan be added to the link without increasing the restoration bandwidthrequirement of the link. In a sense, this metric provides a networkplanner with an idea of the restoration headroom on a link with respectto the proposed primary path. The higher the sharing degree,intuitively, the better the choice of the primary path for the newservice, since a larger sharing degree for a primary path would allowfuture demands to be added along that path without the need to reserveadditional restoration bandwidth.

[0131] Sharing degree can be calculated from the aggregate node-linkvector V_(nla) representation of sharing information and the primarypath node-link vector V_(pnl) representation according to the followingrelationship:

SD=the maximum value m for which max {m·V _(pnl) +V _(nla) }=RB,

[0132] where RB is the current reservation bandwidth on the link underconsideration.

[0133] A less accurate measure of sharing degree can be calculated usingthe compact representations of the aggregate node-link vector. Note thatless accurate means that the sharing degree provides a conservativeindication of which links may be better, but does not provide the exactbandwidth available for sharing for a particular primary path.

[0134] Sharing degree can be calculated from the node-aggregate vectorV_(na) representation of sharing information and the primary path nodevector V_(pn) representation according to the following relationship:

SD=the maximum value m for which max {m·V _(pn) +V _(na) }=RB.

[0135] Sharing degree can also be calculated using the binaryrepresentation of node-link or node vector. In case availability ofsharing information in terms of the binary node-link vector V_(nlb) fora link, the sharing degree can be computed by first deriving a binaryprimary path node-link vector V_(pnlb) from the primary path node-linkvector V_(pnl) (in the similar fashion that V_(nlb) can be derived fromV_(nla)), then taking an AND of the V_(pnlb) (binary primary pathnode-link vector) and V_(nlb) (binary node link vector representation ofthe sharing information on the link), and then taking the bit OR of theresulting vector. If the result is one, sharing is not possible andsharing degree is zero. If the result is zero, then sharing is possibleand sharing degree is one Note that the sharing degree obtained in thismanner using the binary node-link vector does not give the exact amountof sharing that is possible on the link for the primary path. It onlyindicates whether sharing is possible or not.

[0136] Similarly, a more crude sharing degree can be derived by usingthe binary node vector information.

[0137] When bandwidth sharing is possible for a link, it would appearthat there is no immediate bandwidth-related cost for new restorationpath reservation using that link. However, when applying the Markovdecision theory framework described in Dziong '97, there is a cost. Thisfollows from the fact that the cost should be considered during thewhole connection-holding time, not just at the instant of a newconnection arrival. The Markov process makes it possible to consider theprobabilistic cost of using the link, since, even if sharing is possibleat the moment of connection arrival, in the future, with someprobability, the other shared connections can be terminated and the newconnection will be the sole occupant of the reserved bandwidth on thatlink, and hence incur a cost for reserving additional restorationbandwidth in the network. While exact calculation of such a cost seemsto be very difficult, if possible, one can use this argument to employ anon-zero link cost even when sharing is possible. In one implementation,this fact can be accommodated by calculating link cost according to thefollowing equation: $\begin{matrix}{w^{S} = \frac{w^{NS}}{\left( {1 + {b \cdot {SD}}} \right)}} & (2)\end{matrix}$

[0138] where w^(s) is the link cost considering sharing degree, w^(NS)is the link cost without considering sharing degree, and b is aspecified coefficient.

[0139]FIG. 10 depicts an exemplary procedure for calculating a path'scost as a function of link utilization and sharing (assuming the path isa restoration path) as per the above discussion. As illustrated, in step1000, the procedure inputs Path (e.g., the link IDs for a path), Option(Option=1 if the path cost is to be calculated without consideration ofwhether sharing is available or not, and Option=2 if sharing is to beconsidered), and additionally a primary path if Option=2 (since sharingis a function of a path relative to the primary path that it protects).Next, in step 1002 the first link in Path is selected for costassessment. The variable PathCost is initialized to zero in step 1004,and in step 1006, a test is performed to determine if the linkutilization is below a specified utilization threshold. The utilizationthreshold (UT) for the system is determined by analysis and nominallyset to 80%. If the utilization is below or equal to the threshold, thenin step 1008, the link cost is set to AW. If the link utilization isabove the threshold, then in step 1010 the link cost is set according toequation 1. In either case, in the test of step 1012, the variableOption is checked to see if it is set to the no sharing option. If itis, in step 1014, the PathCost is incremented by the link cost. IfOption !=1 (e.g., the “consider sharing” option), then in step 1016 atest is performed to see if sharing is available on this link. If it is,then in step 1018, the link cost is assigned a value according toequation (2) (where it is a function of the link cost calculated for theno-sharing option and the sharing degree calculated using the primarypath that was passed to the procedure) and in either case (whethersharing is available or not on the link) in step 1014, PathCost isincremented by the new link cost. Next, in step 1020, a test isperformed to determine whether all the links in Path have beenprocessed. If they have, in step 1022, PathCost is returned to thecalling procedure, if not, in step 1024, the next link is selected fromPath and step 1006 the processing continues with the link utilizationtest as described previously. The path cost procedure of FIG. 10 canthus be seen to calculate a path cost of a primary path or a restorationpath (if a reference primary path is provided in the latter case). Itcan also be used to calculate a path cost with consideration ofsharability of links in a path (Option=2—only applicable to restorationpaths with respect to a given primary path) or without consideration ofthe sharability of a link.

[0140]FIG. 11 depicts and exemplary method for calculating aminimum-cost primary/restoration path pair for a new service inconsideration of link utilization, utilization threshold, administrativeweight, and sharing degree. As will be seen, FIG. 11 can make use of theprocedure illustrated by FIG. 10 to calculate the path cost in certainsteps. As depicted, in step 1100, the bandwidth and source anddestination IDs for a new service are input. The method involves twomajor flows. In the first flow, illustrated by steps 1102, 1104, 1106,and 1106, one minimum-cost path pair is selected from K candidate pairsby a first method. In the second flow, illustrated by 1110, 1112, 1114,and 1116, one minimum-cost path pair is selected from L candidate pairsby a second method. Then in step 1118, and steps 1120 or 1122, thelowest-cost pair of these two lowest-cost results is selected as theoverall minimum-cost pair.

[0141] Specifically, in step 1102, the first flow starts with selectionof K minimum cost primary paths with the assistance of the PathCostroutine of FIG. 10 using Option=1 (i.e., no sharing considered). Next instep 1104, a set of K minimum-cost restoration paths are calculated andpriced using the assistance of the PathCost routine of FIG. 10 usingOption=2 (e.g., where sharing is considered and the appropriate primarypath is passed to the PathCost routine). Then, in step 1106, thePathCosts that were calculated in steps 1102 and step 1106 are used inthe calculation of a sum of the pair cost for each of the Kprimary/restoration path pairs. In step 1108, the minimum sum is savedin the variable LowKSum and the minimum-cost pair is saved in the datastructure LowKPair.

[0142] In step 1110, the second flow starts with the calculation of Ldisjoint lowest total-cost pairs of paths for the new service where costis determined for each path using the PathCost routine of FIG. 10 withOption=1 (e.g., no sharing considered). Next, in step 1112, the cost ofeach of the L restoration paths that were determined in step 1110 isrecalculated using the PathCost routine, this time with Option=2 (e.g.,sharing considered) and the appropriate primary path passed to theroutine. Next in step 1114, the sum is calculated of the cost of theprimary and the restoration path for each of the L pairs and in step1116, the lowest sum is stored in the variable LowLSum and thelowest-cost pair is stored in the data structure LowLPair.

[0143] Finally, in the test of step 1118, LowKSum is compared toLowLSum. If LowKSum is lower than or equal to LowLSum, then in step1122, LowKPair is selected as the minimum-cost path pair. If it is notlower, then in step 1118, LowLPair is selected.

[0144] Disjoint Path Calculations

[0145] When sharing is not permitted (e.g., when a 1+1 restorationscheme is the best option. available for restoration) or not considereduntil after calculation of the restoration path, the link costs forprimary and restoration paths are considered to be the same, and bothpaths can be calculated using a minimum-cost node-link disjoint optimalalgorithm. This algorithm for N disjoint shortest-path calculations is aspecial case of the “successive shortest-path algorithm,” independentlydeveloped by Jewell, Iri, and Busaker and Gowen. More information on thesuccessive shortest-path algorithm can be found in W. S. Jewell,“Optimal flow through networks,” Technical Report 8, Operations Researchcenter, MIT, 1958; M. Iri, “A new method of solvingtransportation-network problem,” Journal of the Operations ResearchSociety of Japan, 3:27-87, 1960; and R. G. Busaker and P. J. Gowen, “Aprocedure for determining a family of minimum cost networkflow-patterns, Technical Report 15,” Operations Research Office, JohnsHopkins University, 1961, each incorporated herein by reference. Thisalgorithm can be implemented in several different ways with varyingcomplexity. One of the well-known implementations was derived bySuurballe in J. W. Suurballe, “Disjoint paths in a network,” Networks,pp. 125-145, 1974, incorporated by reference herein in its entirety. Aslightly different version of this algorithm is well described in J. W.Suurballe and R. E. Tarjan, “A Quick Method for Finding Shortest Pairsof Disjoint Paths,” Networks, Vol.14 (1984) 325-336, incorporated byreference herein in its entirety, for two disjoint shortest-pathcalculations.

[0146] When sharing is taken into account during the path calculation,the link costs for restoration paths are a function of the primary path(via the sharing degree) as indicated by Equation (2). The situation issimilar when taking into account shared risk link groups (SRLGs). AnSRLG is a set of two or more links, for which a failure of any one linkin the SRLG is associated with a relatively high risk of failure of theother links in the SRLG (e.g., having a risk of failure of the otherlinks in the SRLG greater than a specified risk threshold). For example,when different links correspond to different optical fibers and two ormore optical fibers are physically bundled together, there is a sharedrisk that all of those links will fail together, e.g., as a result of acut in the bundle. Such a set of links may be said to form a shared risklink group. When considering SRLGs, each link that is part of the sameSRLG as a link in the primary path is assigned a cost that is a functionof its membership in that group.

[0147] In general, in calculation of minimum cost primary/restorationpaths for both SRLG and non-SRLG cases, the optimal solution is notfeasible in a reasonable time. Therefore, an efficient heuristic may beused that includes the following steps:

[0148] 1) Create an “intelligent” sequence of potential primary pathsfor a new service (K minimum cost paths and L disjoint path pairs).

[0149] 2) For each primary path, calculate a node-link disjoint (orSRLG-disjoint as specified) shortest restoration path.

[0150] 3) Select the minimum-cost pair of disjoint paths.

[0151] Consideration of Shared Risk Link Groups

[0152] Consider a communications network consisting of nodes and linksinterconnecting those nodes. Each link is at a minimum characterized byits bandwidth expressed in bandwidth units. In general, each connectionor service in the network is carried by a primary path that (1) connectsthe source and destination nodes and (2) provides the necessarybandwidth for the service.

[0153] Primary/Restoration Paths: Disjoint for Protection

[0154] Some of the services can be protected against a single transitnode or link failure. This protection is made possible by selecting aprotection or restoration path for the service that is node and linkdisjoint from the primary path except for the source and destinationnodes. Whenever one of the primary path elements fails, the connectionis switched automatically to the restoration path.

[0155] Primary Paths: Disjoint for Sharing

[0156] With the assumption that a restoration path is selected to bedisjoint from its primary path, it can be seen that the bandwidth of alink that is part of multiple restoration paths can be shared if theprimary paths (that are protected by the multiple restoration paths) arenode and link disjoint from each other. This is because, when theprimary paths are disjoint, a single network element failure is unlikelyto impact two different primary paths. Thus, in the event of a failure,only one restoration path (from the considered set that shares bandwidthon the link) will be activated and only the bandwidth associated withthat restoration path will be allocated.

[0157] Primary/Restoration Paths: SRLG-Disjoint for Protection

[0158] If shared risk link groups (SRLGs) are considered, where there isan increased risk of multiple failures within each SRLG (e.g., disjointlinks that are bundled within the same share a common, a furtherrestriction can be considered. In determining a restoration path for aprimary path, no element (e.g., node or link) of the primary path shouldbe from the same SRLG as an element of the restoration path, since, ifit were, a failure of an element of the primary path might be coincidentwith a failure of an element of the restoration path and recovery wouldbe hampered. Herein, paths that meet this broader type of disjointednesswill be referred to as SRLG-disjoint.

[0159] Primary Paths: SRLG-Disjoint for Sharing

[0160] Similarly, with the assumption that a restoration path is alwaysselected to be SRLG-disjoint from its primary path, it can be seen thatthe bandwidth of a link that is part of multiple restoration paths canbe shared if the primary paths (that are protected by the multiplerestoration paths) are SRLG-disjoint from each other. This is because,when the primary paths are SRLG-disjoint, a single network elementfailure is unlikely to impact more than one of them. Thus, in the eventof a single failure, only one restoration path (from the considered setthat shares bandwidth on the link) will be activated and only thebandwidth associated with that restoration path will be allocated.

[0161] Method of Calculation of Primary and Restoration Paths

[0162]FIG. 12 illustrates one exemplary method for calculating primaryand restoration paths that takes into account bandwidth sharing forrestoration paths and considers the impact of shared risk link groups(SRLGs). Note that this procedure may also be used for calculation ofprimary and restoration paths when no SRLGs are considered byconsidering that each link as an independent SRLG.

[0163] As shown, in step 1200, the source and destination nodes for anew service are input. Next, in step 1202, a set of K shortest-pathcandidate primary paths are calculated (this can be done, for exampleusing the PathCost routine of FIG. 10 with Option=1). Instep 1204, thefirst of the K primary paths is selected for further processing. In step1206, a minimum-cost restoration path is calculated for this primarypath. This calculation could be performed using the routine of FIG. 10(relevant for the non-SRLG case), but alternatively, in this embodimentof the present invention, the procedure illustrated by FIG. 13 ispreferably used, since it specifically addresses link cost as a functionof SRLGs. The procedure of FIG. 13 is discussed in more detail below. Instep 1208, the sum of the primary path cost (calculated in step 1202)and the minimum-cost restoration path cost (determined in step 1206) isstored along with identification of the minimum-cost pair. In step 1210,a test is performed to see if all of the K primary paths wereconsidered. If not, in step 1212, the next primary path from the groupof K is selected and processing returns to step 1206. If, in the test ofstep 1210, it is determined that all the primary paths have beenprocessed, then, in step 1214, all the primary/restoration path pairsthat were saved in step 1208 are analyzed to find the pair with theminimum total cost of primary and restoration paths. The procedure exitsat step 1216.

[0164] While one or more innovations of the present invention are usedto assign a cost to a path, one approach to calculating the candidatepaths themselves that are assigned costs per step 1202 of FIG. 12 isdescribed in Brander, A. W., Sinclair, M. C., “A Comparative Study ofk-Shortest Path Algorithms,” Proceedings of the 11^(th) 1216 PerformanceEngineering Workshop, Liverpool, September 1995, pp. 370, which isincorporated herein by reference in its entirety.

[0165] Calculating the Minimum-Cost Restoration Path for a Primary Path

[0166] As noted earlier, step 1206 of FIG. 12 calculates a minimum-costrestoration path for a given primary path. An exemplary procedure forimplementing step 1206 of FIG. 12 is illustrated by FIG. 13. As shown,in step 1300, the first link of the candidate primary path is selectedand, in step 1302, the shared risk link group(s) to which this linkbelongs are identified. Next, in step 1304, the links in those SRLGs areexcluded from further consideration by either eliminating them from thenetwork graph or, equivalently, assigning an infinite cost to them. Instep 1306, a test is performed to determine whether all the links in theprimary path have been considered. If not, in step 1308, the next linkfrom the primary path is selected and the procedure continues in step1302 with the identification of the SRLGs to which the new link belongs.

[0167] If the test of step 1306 is true, and all the links for thecurrent primary path have already been considered in step 1304, then, instep 1310, the first link of the remaining links in the network isselected for further processing. In step 1312, a test is performed todetermine whether the link's bandwidth can be shared with a restorationpath for the selected primary path. Note that step 1304 guarantees thatthe links considered in step 1312 are already SRLG disjoint from theprimary path. So, as discussed previously, the sharing determinationincludes determining whether the link is already being shared, to whatextent, and by what restoration paths. Note that this determination isassisted by the sharing information that is disseminated to the nodes.If sharing is not possible, then, in step 1314, a test is performed tosee if all the remaining links have been considered for sharing. If not,then, in step 1316, the next link of the remaining links is selected andthe procedure continues with the test of step 1312.

[0168] If the test of step 1312 determines that sharing is possible,then, in step 1318, the cost of the sharable link is reduced bymultiplying the link's current cost by factor R (nominally 1/100) andthe procedure continues with the test of 1314, where the determinationis again made as to whether all the links from the “remaining links” sethave been processed. If they have, then, in step 1320, a minimum-costrestoration path for the primary path is calculated using the new linkcosts. The procedure exits at step 1322.

[0169] Note that, here, bandwidth cost is typically being considered;however, in the more general case, other costs (e.g., administrativeweight and link utilization) can be considered independently or via amultiple criteria function.

[0170] In general, path selection is a powerful tool that can be used toachieve several objectives such as network operator policies (e.g.,preferences for certain links and/or priorities for certain demands),maximization of network resource utilization, and optimization ofdifferent restoration strategies. Each of these objectives is based ondifferent input parameters and optimization models. For example,operator policy can be implemented by means of demand-dependent linkadministrative weights and shortest-path algorithms that minimize theadministrative weight of the path. Maximization of network utilizationcan be achieved by using current link bandwidth utilizations in a pathselection algorithm to achieve load balancing in the network. Differentrestoration strategies can be achieved by modifying network topology andlink-cost functions in the process of path selection in order to achieveoptimal restoration paths that can minimize restoration time and takeinto account SRLGs. One or more of these objectives can be combined intoa unified framework. The framework can be decomposed into three mainparts, each of which is concerned with the calculation of primary andshared restoration paths:

[0171] (1) Choice of the link-state description.

[0172] (2) Choice of the link-cost function used for calculation ofprimary and restoration paths.

[0173] (3) Choice of the algorithm for disjoint paths calculation.

[0174] For the purpose of primary path calculation, the advertisedlink-state description typically will contain administrative weight AW,available capacity AC, and total link capacity LC. This informationsupports calculation of the state-dependent link-cost function, which isthen used to calculate minimum-cost primary paths. For the purpose ofsharing link bandwidth by restoration paths, the node controlling thelink is provided with an accurate but optimized representation of thesharing information.

[0175] Connection Set-Up Protocol Extension

[0176] Historically, protection bandwidth for networks was allocated atthe time of the primary service setup. An example of this is the popular1+1 protection scheme, wherein the bandwidth allocated along theprotection path for a service was the same as the bandwidth allocatedalong the primary path for a service. In this case, the connection setupprotocol was substantially identical at nodes along the primary andrestoration paths, as was the information that was needed at thosenodes. In particular, in order to set up a service connection betweenincoming and outgoing links of a transit node in the prior art, theidentity of the incoming service, identity of the outgoing link, and theconnection bandwidth required by the service were provided.Additionally, procedurally, this same information was provided to nodesalong the primary and protection paths, and the protection bandwidth wasallocated on the protection path at the time of establishment of theservice on the primary path.

[0177] In one or more embodiments of the present invention, however,where sharing of link bandwidth is considered, additional sharinginformation is provided to transit nodes along the protection pathduring a pre-setup of the network. This sharing information is used bythe transit nodes along the protection path to determine the amount ofbandwidth to be reserved along the outgoing protection links of thetransit node to support protection of a new service taking intoconsideration link sharing. Specifically, in the connection setupprotocol, the identity of the incoming service, identity of the outgoinglink, connection bandwidth required by the service, and primary pathlink information are provided to transit nodes along the protectionpath, while transit nodes along the primary path still receive the sameinformation as in the prior art. In one or more embodiments of thepresent invention, the primary path link information can be providedusing the primary path node-link vector V_(pnl) representation, or, lessoptimally, the primary path node vector V_(pn) representation, asdescribed previously.

[0178] In addition, in embodiments of the present invention, bandwidthis not allocated on the links of the protection path, but instead isjust reserved on the links and allocated only in the event of a failurealong one or more of the primary paths that are protected by the links.For this reason, the protection path is often referred to as arestoration path.

[0179] The primary path information, together with the bandwidthrequirement of the new service, provide sufficient information for thetransit node to calculate whether bandwidth sharing is possible and whatthe total bandwidth requirement is for bandwidth reservation on thelink.

[0180] The extension to the service setup procedure that supportssharing in mesh networks is illustrated by the exemplary procedure ofFIG. 14. As shown, in step 1400, a new service request is received. Thismay be at the source node for the service or more generally at adistributed or centralized service or network manager. Next, in step1402, a determination of a suitable primary/restoration path pair ismade for the service. Then, in step 1404, signaling is performed alongthe primary path for the service, wherein the identity of the incomingservice, the identity of the outgoing link, and the bandwidthrequirement of the service are provided to transit nodes on the primarypath. Finally, in step 1406, signaling is performed along therestoration path to achieve reservation of bandwidth for the service,wherein the identity of the incoming service, the identity of theoutgoing link, the bandwidth requirement of the service, and the primarypath link information (e.g., V_(pnl)) are provided to transit nodes onthe restoration path.

[0181] Signaling along the primary and restoration paths can beimplemented using suitable signaling protocols including one or more ofa Craft interface, simple network management protocol (SNMP),transmission control protocol/Internet protocol (TCP/IP) directconnections, PNNI, RSVP-TE, and constrained route-label directed path(CR-LDP) protocol.

[0182] OSPF Exemplary Implementation

[0183] The principles described in this specification can be applied toa broad class of different networks by working within defined protocolsfor those networks and/or providing extensions to existing protocols forthose networks. For example, sharing may be realized within Internetprotocol (IP) networks by appropriate extensions to the protocols usedwithin IP networks for distribution of link-state information andsetup/reservation of paths.

[0184] As a specific example, a popular dynamic routing protocol used inautonomous systems built from IP networks is open shortest path first(OSPF). OSPF includes mechanisms for determination and distribution oflink-state information. The distribution includes both time-based andtrigger-based flooding mechanisms that support both full and incrementaldistribution of link-state information between nodes in the network.More information on OSPF can be found in Moy, J., “OSPF version 2,”RFC1247, July 1991, incorporated herein by reference in its entirety.

[0185] Recent extensions to OSPF include the opaque link stateadvertisement (LSA) option. This option effectively provides for the useof the OSPF flooding mechanism to distribute optionalapplication-specific information in the OSPF IP messages that areflooded through a network. By defining an application-specific type forsharing information, the opaque LSA option allows sharing information,preferably in a compact and efficient format (e.g., node-link vector ornode vector) to be distributed throughout the network and reach thevarious nodes that control the reservation and allocation of bandwidthon protection links within the network. The OSPF LSA option is describedin Coltun, R., RFC2370—OSPF Opaque LSA Option, Network Working Group,July 1998, incorporated herein by reference in its entirety.

[0186] The sharing information that has been distributed to the nodes inthe network can be used to calculate, reserve, and allocate bandwidth onlinks. Popular protocols used in IP networks for establishment of paths(also known as “tunnels” in the language of the RFCs that describe thoseprotocols) are RSVP and RSVP-TE. RSVP and RSVP-TE are described indetail in R. Braden, Ed., et al., “Resource Reservation Protocol(RSVP)—version 1 functional specification,” RFC2205, September 1997, and“D. Awduche, et al., “RSVP-TE: Extensions to RSVP for LSP tunnels”,RFC3029, December 2001, respectively, each incorporated herein byreference in its entirety.

[0187] Shared Restoration Extensions to OSPF

[0188] The following describes how the concepts of shared restorationmay be applied to IP networks by appropriate extension of theOSPF/OSPF-TE protocols. These extensions allow OSPF/OSPF-TE todisseminate sharing information to nodes in an IP network and enablenodes to identify, for each new service, an optimal combination of aprimary path and a restoration path, from source to destination, suchthat the amount of bandwidth reserved on the restoration path isminimized.

[0189] The extensions described herein assume an IP network with trafficengineering (TE) support such as that described by the multiprotocollabel switched (MPLS) umbrella of protocols. More background informationon MPLS-based recovery can be found in Sharma, V., Hellstrand, F., etal., “Framework for MPLS-based Recovery,” IETFdraft-ietf-mpls-recovery-frmwrk-07.txt, September 2002 incorporatedherein by reference in its entirety. More background on shared meshprotection under MPLS can be found in Qureshi, M., et al., “MPLS-TEShared Mesh Protection,” IETFdraft-qureshi-mpls-shared-protection-00.txt, October 2002.

[0190] The extensions described herein address the issues of supportingshared mesh protection in mesh networks that is guaranteed for onefailure at a time. It is assumed that the restoration path is disjointfrom the primary path and is pre-computed and reserved at the time ofthe primary path setup. Bandwidth is reserved but not allocated prior toan actual failure. Once the failure of a primary path is identified, theprotection path is then activated by allocating the reserved bandwidth,thereby supporting fast restoration. By reserving the bandwidth inanticipation of a failure but not allocating the bandwidth until thefailure actually occurs, it is possible to share the reservedrestoration bandwidth on a particular link among multiple otherconnections that do not have common elements (links, nodes) in theirprimary paths (other than the common source and destination nodes).

[0191] The pre-computation of the protection path can be done in one ofseveral ways, depending on the available information. One way is tocalculate a disjoint path using the same link weights as those used inthe calculation of the primary path. In this case, one can use RSVP-TEextensions for shared mesh protection to compute locally, at every nodealong the protection path, the amount of shared protection bandwidthrequired on each of the links that constitute the restoration path. Moreinformation on this approach can be found in Liuh, H. A., et al.,“RSVP-TE Extension for Shared Mesh Protection,” IETFdraft-liu-rsvp-mpls-shared-protection-00.txt, October 2002 (“Liuh '02”),incorporated herein by reference in its entirety. While some sharing ofprotection bandwidth may be realized in this approach, the amount ofbandwidth reserved for recovery is typically not minimized since thesharing potential on each of the links is not known before theprotection path choice is made at the source node.

[0192] In contrast, the proposed extensions to OSPF-TE provide forsufficient information to be disseminated to each node to supportefficient restoration path calculation and a significant improvement innetwork utilization.

[0193] Given a primary path between a source-destination pair, there area couple of different methods to compute the optimal restoration path.One way is to compute ‘N’ disjoint paths (with respect to the givenprimary) and then start a resource signaling procedure (throughprotocols like RSVP) to determine the path that would result in theleast amount of bandwidth being reserved on each hop (link) of theprotection path. Extensions to RSVP-TE have been proposed to allow forthis computation; however, those extensions involve sending signalingmessages across each of the nodes that constitute a potentialrestoration path, with likely crank-backs. In contrast, the proposedextensions to OSPF-TE make it possible to compute a primary andrestoration path combination that utilizes the network resourcesefficiently with reduced signaling and a reduced crank-back load. Theseextensions to OSPF-TE are meant to work alongside the RSVP-TE extensions[Liuh '02] While the latter provides the minimal topological informationneeded for guaranteed restoration, the former allows for efficient useof network resources.

[0194] The information made available to network nodes by theseextensions can be used to build an extended link-state database just asnode link-state advertisements are used to build a “regular” link-statedatabase. The difference is that the extended link-state database(referred to below as the TE/Share database) has additional linkattributes. Uses of the TE/Share database include (1) monitoring theextended link attributes and (2) globally optimized computation ofprotection paths.

[0195] In this scheme, in order to support sharing/restoration, everynode maintains a local sharing information database (e.g., the TE/Sharedatabase). This database contains a data structure for each link L thatthe node owns. The data structure for each link contains information(e.g., bandwidth, link-id) about all the other links in the network forwhich link L provides restoration capacity. Since this could be a largeamount of information for a large network, only changes in this databaseare advertised. Note that, in practice, links are unidirectional. Byconvention, the node that owns a link is the one that is connected tothe outgoing interface side of the link, and that node is the one thatmakes advertisements about the link.

[0196] As an example, let link L be an outgoing link from node R, wherelink L is part of a restoration path for a primary path of bandwidth B1,which passes through links L1, L2, and L3. Assuming that link L is not apart of any other restoration path, then the advertisements for link Lfrom node R would be B1::L1,L2,L3. This advertisement says the bandwidthreserved on link L with respect to a failure of link L1, L2, or L3 isB1. If link L is subsequently used to provide restoration for anotherprimary path of bandwidth B2, which passes through links L2, L4, L5,then the advertisements for link L from node R would then be B1+B2::L2;B2::L4,L5. Note that the new advertisement contains no information forlinks L1 and L3, since the protection bandwidth reserved on link L withrespect to those links did not change. Note that B1+B2 bandwidth is nowrequired to protect against the failure of link L2, since two primarypaths share link L2, one of bandwidth Bi and one of bandwidth B2.

[0197] Each time a link is used as a restoration link, and the requiredshared-mesh-protection bandwidth is reserved, the TE/Share database isupdated to reflect the changes if any that are necessary. Note that,although it is preferable for the TE/Share database to be different fromthe LSA database, it is possible to use a common database. However,since it is desirable to keep the LSA advertisements as small aspossible, only the changes to the TE/Share database are advertised. Sucha restriction might require a separation of the databases on most OSPFimplementations.

[0198] Although the extensions proposed herein do not specify how tocompute primary or restoration paths, they do provide information usefulin making an optimal restoration path choice (with respect to a givenprimary path) from a network utilization standpoint.

[0199] In one implementation, the primary path is provided as a list ofnode-ids (i.e., the path is specified by the nodes through which itpasses) by a route-database. However, to find out if indeed a primarypath can be supported through the specified node-ids, and to determinethe exact links through which it traverses, a signaling protocol such asthe proposed extension to RSVP-TE [liuh '02] could be used.

[0200] Sharing

[0201] When a new service is added to a network, the source node or anetwork management entity has the task of determining appropriateprimary and restoration paths for that service. Assuming a set ofcandidate primary paths can be calculated for the service as well as aset of potential restoration paths for one or more of those primarypaths, it is then of interest to determine, for each candidate primarypath, a restoration path that offers the best sharing opportunity. Onemetric that can be used to evaluate a restoration path is the overallamount of additional bandwidth that must be reserved in the network tosupport protection of the new service using that restoration path. Bychoosing a restoration path that would require the minimum amount ofadditional bandwidth to be reserved along its links, an operator canreduce the overall impact to the available capacity in the network.

[0202] Assume that a candidate primary path P of bandwidth B_(p) ischosen for a new service. Assume further that a number of candidaterestoration paths R₁, R₂, R₃, . . . , R_(N) are also chosen that aredisjoint from primary path P. Let P-set be the set of all links thatconstitute primary path P. Let Rj-set denote the set of links Li (i={1,. . . , m} where m is the number of links in Rj) that make uprestoration path Rj. Next, let B-Li-set denote the set of links in thenetwork for which link Li already provides restoration capacity (wherefor B was chosen here as a mnemonic for “backup”). With theseassumptions, the following observations can be made:

[0203] (a) No sharing possible: If any B-Li-Set is null, then thecorresponding link Li is not already providing restoration capacity forany link in the network. Since it is not already backing up any links,it does not currently have any reserved restoration bandwidth to share;and, if it were to be used as part of a restoration path for primarypath P, then it would be necessary to reserve the full amount of theprimary path's bandwidth B_(p) on link Li. In this case, the “cost”assigned to this link of the restoration path under consideration wouldbe Bp. On the other hand, if B-Li-set is not null, then some sharingpotential exists on link Li, but it still needs to be seen whether thissharing potential applies to any of the links of the primary path underconsideration.

[0204] (b) Sharing possible—scenario A: For non-null B-Li-sets, it is ofinterest to see if any of the links that are protected by Li are in theprimary path. If the intersection, denoted I-Li-set, between P-Set andthe B-Li-Set for a link Li is null, then link Li is not providingprotection for any of the primary path links, but it is providing somelevel of protection for links in other primary paths in the network thatare disjoint from the primary path of interest. Thus, by definition,some protection bandwidth sharing is possible on link Li. There are twocases.

[0205] In the first case, the bandwidth Br(i) that is presently reservedon link Li is greater than or equal to Bp. In this case, no additionalbandwidth need be reserved to protect the new service, and the cost ofusing this link of the candidate restoration path is zero.

[0206] In the second case, Bp is greater than the presently reservedbandwidth Br(i) on link Li, and the incremental bandwidth that needs tobe reserved is Bp-Br(i). If, however, Bp is greater than the totalcapacity of the link, then the link cannot be used to protect theservice, and the cost of link Li is preferably set to a sufficientlyhigh value to exclude this link and, in fact, this entire candidaterestoration path, from further consideration as a restoration path forthis service on this primary path. Alternatively, this “total capacity”constraint may be applied to exclude such paths from being included whenthe set of candidate restoration paths is originally generated.

[0207] (c) Sharing possible—scenario B: Finally, if the intersectionI-Li-set between B-Li-set and P-set is non-null, then the intersectionindicates which links of the candidate primary path are alreadyprotected by link Li for other primary paths. If a link Lk (fromI-Li-set) is already protected by link Li, then the total amount ofbandwidth that will need to be reserved on link Li to protect against afailure of link Lk is at least the amount of protection bandwidth Bkneeded on link Li to protect other services on link Lk plus thebandwidth Bp of the new service. There are again two cases.

[0208] In the first case, the bandwidth Br(i) that is presently reservedon link Li is greater than or equal to Bp+Bk for all links Lk inI-Li-set. In this case, no additional bandwidth need be reserved toprotect the new service, and the cost of using this link of thecandidate restoration path is zero.

[0209] In the second case, Bp+Bk is greater than the presently reservedbandwidth on link Li for at least one link Lk in I-Li-set, and theincremental bandwidth that needs to be reserved is Bp+max(Bk)|_(k)−Br.If, however, Bp+max(Bk)|_(k) is greater than the total capacity of thelink, then, the link cannot be used to protect the service and the costof link Li should be set to a sufficiently high value to exclude thislink and, in fact, this entire candidate restoration path, from furtherconsideration as a restoration path for this service on the consideredprimary path.

[0210] The above information is consolidated into an exemplary procedurefor determining a minimum-cost restoration path given a particularprimary path as illustrated by FIG. 15. As shown, in step 1500, aprimary path is selected for a new service of bandwidth Bp, and the setof links P-set on that primary path is initialized. Next, in step 1502,N candidate restoration paths are chosen from amongst the possiblealternative paths in the network that are disjoint from the primarypath. In step 1504, restoration path index j is initialized to zero, andthe variables Lowest_Cost and BestRPathID are initialized to asufficiently high value (e.g., infinity) and 0, respectively. Next, instep 1506, the cost Cost_Rj of the restoration path is calculated as asum of the costs of its constituent links (more detail is shown in FIG.16 described below), and the calculated Cost_Rj is compared with thecurrent value of Lowest_Cost in the test of step 1508.

[0211] If Cost_Rj is less than Lowest_Cost, then, in step 1510,Lowest_Cost is set equal to Cost_Rj, and BestRPathID is set equal torestoration path index j. Next, or if the test of step 1508 fails, instep 1512, index j is incremented, and the test of step 1514 is thenperformed to determine whether all the candidate restoration paths havebeen considered. If they have, then the procedure finishes in step 1516.If they have not, then the procedure continues with the calculation ofthe cost of the next candidate restoration path in step 1506.

[0212]FIG. 16 illustrates an exemplary procedure for calculating thecost of each restoration path Rj per step 1506 of FIG. 15. The procedurebegins in step 1600 with the initialization of Rj-set={links incandidate restoration path Rj}. Next, in step 1602, for each link Li inRj-set, B-Li-set={links protected by Li}. Note that this protectioninformation is stored locally, centrally, or in distributed fashion atone or more nodes in the network and may be communicated completely orincrementally and stored via one or more of the mechanisms discussedelsewhere in this specification (e.g., OSPF flooding mechanism and OSPFopaque LSA and TE/Share database). In step 1604, for each link Li,I-Li-set={intersection of B-Li-Set and P-set}. Next, in step 1606, linkindex i and variable Cost_Rj are initialized to zero, and, in step 1608,index i is incremented.

[0213] In step 1610, a test is done to see if B-Li-set is nullindicating the lack of any sharing potential in link Li. If the set isnull, then, in step 1612, Cost_Li is set equal to the bandwidth Bp ofthe primary path. Otherwise, in step 1614, as test is performed todetermine whether the intersection set I-Li-set is null indicating nocommon links between the primary path and those links being protected bylink Li.

[0214] If the intersection set is null, then, in step 1616, Cost_Li isset equal to the maximum of either zero or the result of subtracting thebandwidth Br(i) that is already reserved on link Li from the bandwidthBp of the new primary path service. If the intersection is non-null,then, in step 1618, Cost_Li is set equal to the maximum of zero andBp+max|_(k)(Bk)−Br(i).

[0215] Following steps 1612, 1616, or 1618, the test of step 1620 isperformed to determine whether the additional bandwidth, if any,required on link Li exceeds the available capacity of link Li. If itdoes, then, in step 1622, Cost_Rj is set to a sufficiently high value(e.g., infinity), and the procedure exits in step 1624. Otherwise, instep 1626, the cost of the current link is added to the current cost ofthe restoration path, and the procedure continues to step 1628, whichdetermines whether all of the links in the current protection path Rjhave been considered. If not, then the processing returns to step 1608;otherwise, processing exits at step 1624.

[0216] As an example, consider the partial network 1700 of FIG. 17.Consider the primary path L1-L2 with four units of bandwidth. Also,assume that there are two possible disjoint routes (with respect to theprimary path) to the destination, namely route A (L3-L4) and route B(L5-L6). Further, assume that the entries of the current TE/Sharedatabase (DB) are as shown below:

[0217] Entry for L3 in TE/Share DB→4::L1, L8; 5::L10

[0218] Entry for L4 in TE/Share DB→4::L8, L9

[0219] Entry for L5 in TE/Share DB→4::L9, L10

[0220] Entry for L6 in TE/Share DB→4::L8, L10

[0221] The P-set for this example is {L1, L2}, the first R-set,R_(A)-set {L3, L4} and the second R-set, R_(B)-set={L5, L6}. TheB-Li-sets for restoration path A are B-L3-set={L1, L8, L10} andB-L4-set={L8, L9}. Similarly, the B-Li-sets for restoration path B areB-L5-set={L9, L10}, B-L6-set={L8, L10}. Evaluating the first restorationpath, it is noted that none of the B-Li-sets are null, thus each of thelinks within the first restoration path offers some potential forsharing. Here, for example, the entry for link L3 indicates that L3needs to reserve four units of bandwidth to protect L1 and L8 and fiveunits of bandwidth to protect L10. Note that links L8, L9, and L10 areoutside of the partial network 1700 shown in FIG. 17, but are stillconsidered in the above example to levy protection requirements on thelinks as indicated below. Note that the actual reserved bandwidth onlink L3 would be the maximum of those two bandwidths, namely five units.Similarly, the entry for L4 indicates that L4 needs to reserve fourunits of bandwidth to protect against failures on L8 and L9. The entriesfor links L5 and L6 are interpreted similarly.

[0222] In operation, source node R1 would compare the primary path L1-L2against the entries for every link in a restoration path (e.g.,R_(A)-set={L3, L4}) and repeat the comparison for all possiblerestoration paths (e.g., R_(A)-set and R_(B)-set). Upon comparison withpath L3-L4, it is found that link L1 is already shared and that, if nodeR1 were to choose this restoration path, then the reservation bandwidthon L3 would need to be increased by 3 (i.e., 4+4−5) additional units ofbandwidth. However, no additional bandwidth would need to be reserved onL4.

[0223] Upon comparison with restoration path L5-L6, it is found that theintersection of P-set {L1, L2} and R_(B)-set {L8, L9, L10} is null, andno additional units of bandwidth would need to be reserved (since 4units have been reserved already). Therefore, under such circumstances,the restoration path L5-L6 would be considered to be the betterrestoration path.

[0224] On the other hand, if the entries corresponding to L3 and L4remain unaltered, but those corresponding to L5 and L6 have changed tothose below:

[0225] Entry for L5 in TE/Share DB→1::L9, L10

[0226] Entry for L6 in TE/Share DB→1::L8, L10 then it can be seen thatchoosing restoration path L5, L6 would require a total of six additionalunits of bandwidth (i.e., three additional units for each of links L5and L6) to support a primary path of four units of bandwidth. In thiscase, path L3-L4 is a better choice for restoration since only a totalof three additional units of bandwidth need to be reserved in thenetwork.

[0227] LSA Formats for Sharing

[0228] The Shared Restoration LSA starts with the standard LSA header.The LSA payload includes one or more nested Type/Length/Value (TLV)triplets for extensibility. The Length field defines the length of thevalue portion in octets (a TLV with no value portion would have a lengthof zero). The TLV is padded to a four-octet alignment. Padding is notincluded in the length field (so a three octet value would have a lengthof three, but the total size of the TLV would be eight octets). NestedTLVs are also 32-bit aligned. Unrecognized types are ignored. All typesbetween 32,768 and 65,535 are reserved for vendor-specific extensions.All other undefined type codes are reserved for future assignment by theInternet Assigned Number Authority (IANA). A Shared Restoration LSAcontains one top-level TLV. The Restoration TLV describes a single linkof the advertising node. Only one Restoration TLV is be carried in eachLSA. The Link TLV is type 1; the length is variable and describes thelength of the value field. The Link Type (1 octet), Local InterfaceIndex (4 octets), and Local Interface IP Address (4 octets) should benon-zero, while the value of the reserved field should be zero and isreserved for future use. All other fields defined are presentconditional to the value of the Resource Flag field.

[0229] The Link Type field defines the type of the link: 1 impliespoint-to-point, 2 implies multi-access. The Local Interface IP Addressfield specifies the IP address of the interface corresponding to thislink. This field is 4 octets in length. The format described supportsIPV4 addressing, and the reserved fields can accommodate IPV6. TheReserved field can be used to specify this, and the length of the TLVcan be appropriately increased. The first octet of the value field, theResource Flag, describes the kind of resources that follow. If ResourceFlag is set to 0×01, only the restoration bandwidth and the maximumrestoration bandwidth are present, where restoration bandwidth is thecurrently allocated restoration bandwidth on that link and the maximumrestoration bandwidth is the maximum amount of restoration bandwidthallowed on that link. Both the Restoration Bandwidth and the MaximumRestoration Bandwidth fields are expressed in IEEE floating-point formatand specified in bytes.

[0230] If the Resource Flag is set to 0×10, then the RestorationBandwidth and the Maximum Restoration Bandwidth fields are absent. Whatfollows in the LSA are the various primary links that this link providesrestoration for, and the bandwidth on each of them. The Bandwidth fielddescribes the amount of bandwidth in IEEE format, in bytes. The PrimaryLinks fields following the Bandwidth field are the primary links forwhich this link provides restoration, and whose bandwidth on each ofthem is specified by the Bandwidth field. Once again, only IPV4addresses are illustrated. The Reserved field may be used in the futureto indicate IPV6 addresses. The Number of Primary Links field describesthe number of primary links that follow the Bandwidth field.

[0231] If the Resource Flag is set to 0×20, once again the RestorationBandwidth and Maximum Restoration Bandwidth fields are absent. Whatfollows in the LSA are the various primary links that this link providesrestoration for. This case is similar to the case when Resource Flag isset to 0×10, except that this case addresses the issue of findingbandwidth sharing, rather than the actual amount of bandwidth that isshared. In a way, this is a binary form of the previous case, whereinthe problem of “can sharing be done” is addressed in a true-or-falseform. Once again, all IP addresses are IPV4, and the Reserved field maybe used in the future to indicate IPV6 addresses.

[0232] Cases where the Resource Flag is set to 0×11 and 0×21 are similarto cases 0×10 and 0×20, respectively. However, the Restoration Bandwidthand the Maximum Restoration Bandwidth fields are also present. Notethat, with Resource Flag set to 0×11, the source node would havedetailed information about the constraint and degree of sharing of linkrestoration bandwidths during the computation of restoration path. Useof other modes trades away the network utilization with less informationfor advertisement and local bookkeeping.

[0233] A summary of one possible structure of link-state advertisementper opaque LSA extensions is as follows:

[0234] Fields: Intf |N-id|Flag|AW|LC|AC|D|U-SN|Len|Rid|Rid|. . . |Rid|where the fields in the LSA are as follows:

[0235] Intf—the interface number (in its node) of this link (16 bits)

[0236] N-id—the neighbor's IP address (32 bits)

[0237] Flag—determines what follows in the rest of the LSA

[0238] AW—the administrative weight of the link (16 bits)

[0239] LC—the bandwidth capacity of the link (16 bits)

[0240] AC—the bandwidth that is actually available for use (24 bits)

[0241] U-SN—the update sequence number (32 bits)

[0242] Len—number of Rid that follows in this LSA (8 bits)

[0243] Rid—id of a node, for which the binary node vector value is 1 (32bits)

[0244] Advertising only non-zero elements of the node-binary vector canreduce significantly the amount of the advertised data

[0245] Use and Handling of the Shared Restoration LSAs

[0246] Nodes originate Shared Restoration LSAs whenever the TE/Sharedatabase changes and whenever otherwise indicated by OSPF (an LSArefresh, for example). Upon receipt of a changed Shared Restoration LSA,a node will update its TE/Share database. No shortest path first (SPF)or other route calculations are necessary. Interoperability with nodesthat do not implement the extensions should not have problemsinteroperating with those that do, as the Opaque LSAs will be silentlyignored. The result, however, will be that the network might not be ableto utilize the resource as efficiently and may fail to find arestoration path when there is a viable one.

[0247] Reservation of the Protection Capacity Pool

[0248] The protection capacity reserved on a given link is also known asa capacity pool because it may be shared among multiple paths forprotection purposes. In the embodiment of SMDN 100 of FIG. 1, this poolmay be either statically provisioned or dynamically assigned. In bothcases, an MPLS switch located at each node in the SMDN is capable ofsetting aside capacity on each of its connected links without actuallyallocating it to any LSPs. In practice, each protection LSP is set upwith zero or some minimum bandwidth to support operation,administration, and maintenance (OAM) packets associated with theprotection signaling of the network. This bandwidth is typically smallerthan the bandwidth reserved to protect the primary LSP traffic.

[0249] In case of static provisioning, a pool of bandwidth on each linkis set aside in advance of service provisioning specifically for thepurpose of recovering traffic affected by a failure. If the amount ofprotection bandwidth is known in advance, then static allocation becomesan option. For example, in the case where the SMDN to be protected has aring topology, half of the available network bandwidth might bestatically allocated in advance as the protection bandwidth. Independentof what combination of services of different bandwidths get added to theworking half of the bandwidth, the protection half of the bandwidthshould be sufficient to protect all working traffic affected by a singlelink or node failure in the network. By using static allocation, inapplying shared mesh protection methodology, bookkeeping is minimized.The call admission control (i.e., the process that accepts a new call ordata communication request into the network) for the working traffic(LSP) typically will only admit those calls that will be sufficientlyprotected by the available bandwidth along the protection path (given atmost a single failure in the network). In the case of dynamicassignment, the size of the protection pool (i.e., the protectionbandwidth reserved on each link of the network) may dynamically expandor shrink based on the protection needs of the network as newconnections (LSPs) are being admitted or established, and existingconnections (LSPs) are being removed, respectively. Thus, dynamicassignment provides flexibility to adapt the protection to changes inthe traffic.

[0250] In a distributed implementation of the shared mesh data network,when sharing information is distributed across the nodes in the network,each node along a new protection path computes the amount of additionalprotection bandwidth it might need on its downstream connected linkalong the new protection path to protect a corresponding new primaryLSP. For additional protection bandwidth, a node first checks for theavailability of the additional bandwidth on the connected link, and, ifthe additional bandwidth is available, then the node adds the additionalbandwidth to the reserve protection pool of the connected link that isdownstream of the node along the protection path.

[0251] In case of the centralized server approach, a centralized servercomputes the additional protection bandwidth needed on each link alongthe protection path to admit a new connection (LSP). This information isthen conveyed as part of network signaling to establish the protectionLSP.

[0252] Fast Failure Detection and Notification

[0253] As in other asynchronous transport networks, an SMDN thatincorporates MPLS for traffic engineering might suffer from uncertaintyand potentially unbounded delay in the delivery of packets. Thissometimes makes it more difficult to achieve fast failure detection andnotification. This is because a failure might not be detected bymonitoring just the absence of packet arrivals at a destination.Specifically, there may be ambiguity in distinguishing between theabsence of packets due to a failure of an LSP and the absence of trafficon the LSP due to a lull in communication between the source anddestination. Therefore, to achieve fast failure detection, in certainimplementations of an SMDN that utilizes MPLS for traffic engineering,any of four alternative approaches might be used.

[0254] In the first approach, failure detection relies on the physicaltransport (optical layer) to detect and propagate failure indications upinto the MPLS layer. In this approach, every node in the network isassumed to be capable of propagating failure indications downstream byinserting forward-defect-indicator operations-and-maintenance (FDI-OAM)packets into the flow of traffic. More details on OAM and FDI-OAM inMPLS networks can be found in ITU-T Recommendation Y.1711, “OAMmechanism for MPLS networks,” November 2002 (herein Y.1711),incorporated herein by reference in its entirety. In this approach, assoon as a node that is downstream from a physical-layer failure detectsthe failure, it inserts FDI-OAM packets carrying the failure indicationin all affected LSPs passing through it. Note that these FDI-OAM packetsare inserted in-band in the downstream direction. The destination nodeof each of the affected primary paths will ultimately receive thesespecial OAM packets. Upon receiving this in-band failure notification,the destination passes the failure information to the source of theaffected LSP using backward-defect-indicator OAM (BDI-OAM) packets (alsodefined in Y.1711). Note that a failure indication from the destinationto the source is passed using a pre-established LSP that is disjointfrom the failed primary LSP.

[0255] Using this first approach, the SMDN can recover quickly (e.g., onthe order of tens of milliseconds) from failures that are detectable atthe optical layer. However, there may be other failures above theoptical layer that can cause failure of LSPs. Since these failures mightnot be detectable at the optical layer, optical-layer detection alonemight not provide full coverage for failure monitoring of MPLS paths(LSPs).

[0256] In a second approach, failure detection and protection at the LSPlevel can be achieved by periodically inserting special OAM packets,known as fast failure detection (FFD) packets, into the primary LSPs ofthe SMDN. When a working LSP is provisioned, the source node of the LSPis configured to generate and insert FFD-OAM packets periodically intothe LSP with a time interval T. At the receiver side, the arrival ofFFD-OAM packets is continuously monitored. The receiver registers afailure on the LSP when it does not see any FFD-OAM packets on the LSPfor an interval of nT, where n is a configurable integer value (n≧1).Note that a larger value of n reduces false failure detectionprobability.

[0257] False failure detection occurs when packet delays or loss ofFFD-OAM packets (rather than a true connection failure) result inincorrect declaration of an LSP failure. This problem can be addressedwithin the packet quality-of-service (QoS) framework. For example, onecan mark the label of these packets in such a way that these packets aretreated with high priority in scheduling and buffer management at eachlabel-switched node within each node of the LSP. Consequently, delayjitters and dropping of FFD-OAM packets are minimized. QoSprioritization can also be applied to FDI-OAM and BDI-OAM packets toimprove performance of the first approach, which was based on failureindication at the optical-transport layer.

[0258] In a third approach, the source node of an LSP inserts FFD-OAMpackets when the LSP is idle. Specifically, when the LSP is idle formore than a specified time interval T an FFD-OAM packet is inserted andsent to the destination node. The destination node will declare afailure of the LSP when the LSP is in the idle state for an intervalgreater than nT where n and T are provisionable parameters of thenetwork. The value of n should be selected such that false failuredetection probability is minimized.

[0259] Finally, in a fourth approach, a combination of an optical-layerdetection mechanism (e.g., the first approach) and a higher-levelMPLS-layer detection mechanism (e.g., the second or third approach) isused to achieve fast and comprehensive failure coverage and recovery.

[0260] Assigning Reserve Capacity to Protection LSPs

[0261] It is assumed that every protection LSP is established with anassigned bandwidth of zero or some minimal value sufficient to carry OAMtraffic. After a failure, traffic coming into the ingress node for aprimary LSP is switched to its corresponding protection LSP, and trafficexiting the egress node for the primary LSP is selected from theprotection LSP, instead of from the primary LSP. The aggregate bandwidthfor the protected traffic on primary LSPs is reserved in advance as partof the protection pool in the SMDN. Note that the bandwidth of theprotection pool is not reassigned to any protection LSP. After thefailure, each protection LSP that is providing protection against aspecific failure needs to be assigned the same bandwidth and QoScharacteristics as were assigned to the primary LSP it protects. Thisrequires changing the bandwidth assignment of each of these protectionLSPs. This can be accomplished through RSVP-TE using its bandwidthchange procedure (see RFC 3209 for details).

[0262] Switching Between Working and Protection LSPs

[0263] As discussed above, after detecting a failure, end nodes of anLSP switch traffic from a primary (i.e., working) LSP to itscorresponding protection LSP. Note that both working and protection LSPstypically enter the ingress node and exit the egress node usingdifferent ports.

[0264] At the ingress node, this may be accomplished by the nodemaintaining two different next-hop label-forwarding entry (NHLFE)entries in the MPLS forwarding equivalence class-to-NHLFE (FEC-to-NHLFEor FTN) map (one for the working path and the other for protection path)and activating only one of these at a time. Since the FTN is used to mapincoming client packets to an NHLFE, activating one or the other ofthese entries will serve to direct packets coming into an ingress nodeto the working or the protection path, effectively switching betweenworking and protection LSPs. Thus, protection switching on ingressinvolves switching from the working to the protection NHLFE.

[0265] On the egress node, there is a separate NHLFE entry for workingand protection LSPs dictating the next action. Based on the label (for aswitch-wide label-space implementation) or combination of port and label(for a port-based label-space implementation) that the packet carries,the proper entry is chosen and correspondingly the next action/operation(e.g., pop the label and pass the payload to the client layer) (see RFC3031) is dictated by the entry for the packet. Thus, no specificswitching really needs to be performed at the egress node since theaction has the effect of delivering the data from the proper path (e.g.,primary or protection) to the client. Note that both entries normallydictate the same next action.

[0266] Functional Architecture

[0267] An SMDN can be partitioned into a variety of modules. Thesemodules are part of a network manager that may be implemented at acentralized server or distributed partially or fully to the nodes. Someexemplary modules, representing one specific functional partitioning ofthe components of an SMDN, which provide functionality such as faultdetection and notification, protection switching, signaling and control,routing, and OAM, are listed below:

[0268] Service Agent (SA): This module provides interface to applicationor service management for provisioning paths with differentrestoration/QoS requirements.

[0269] Path Management (PM): This module primarily consists of thesignaling protocol such as RSVP-TE. It is responsible for path setup,refresh, tear-down, and monitoring functions.

[0270] Path Computation (PC): This module provides path computationservices to the path management module. It interfaces with thelink-state-based routing protocol to obtain the network topology.

[0271] Network Topology (NT): This module primarily consists of thelink-state-based routing protocol. It is responsible for monitoring andupdating of network topology.

[0272] Fault Detection (FD): This module provides the fault detectioncapability. Faults can be detected through monitoring of layer-1,layer-2, and layer-3.

[0273] Fault Notification (FN): This module is responsible for notifyingprotection switching nodes of detected faults.

[0274] Protection Switching (PS): This module is responsible forswitching the affected traffic onto a protection LSP after detecting afailure or receiving failure notification.

[0275] OAM&P (OAM): This module is responsible for providing performancemonitoring functions on a per-LSP basis through use of special OAM&Ppackets. It can help in detecting any faults affecting the MPLSshim-layer.

[0276] Bandwidth-Sharing Information (BSI): This module is responsiblefor managing the sharing of information related to protection sharing inthe SMDN. It provides functionality at each node to keep track of thesharing status at each of the incident links to that node. This moduleworks in conjunction with the PM module, which uses special signaling tocarry working path information (bandwidth, nodes, and links) along theprotection path in the SMDN. This information is required to keepbandwidth-sharing information current.

[0277] These functional modules can be classified into two groups. Thefirst group includes those functional modules that support a manuallyprovisioned shared mesh protection service. This set includes the FD,FN, PS, and OAM modules. The second group includes modules that bringautomation to resource discovery, path computation, and pathestablishment and management. These modules can perform signaling andcontrol plane functions. They include the SA, PM, PC, and NT modules.

[0278]FIGS. 18, 19, and 20 illustrate the functionality of the ingress,intermediate, and egress nodes, respectively, of the SMDN. These figurescollectively illustrate a distributed implementation of the pathcomputation function of SMDN 100 of FIG. 1. In contrast to theillustrated implementation, in a centralized implementation, pathcomputation and sharing information functions would reside on acentralized server.

[0279] Ingress, Intermediate, and Egress Nodes

[0280]FIGS. 18, 19, and 20 depict the interrelationship of the subset ofmodules that comprise ingress, intermediate, and egress node operationsof the nodes within an SMDN according to one embodiment of the presentinvention. Note that, typically, any node within an SMDN may at one timeor another, for one service or another, serve as an ingress node, anintermediate node, or an egress node, or simultaneously as two or moreof ingress, intermediate, or egress nodes. Therefore, FIGS. 18, 19, and20 do not necessarily depict unique nodes in the SMDN, but ratherhighlight the relevant functions of a node that are associated with itsoperation as ingress, intermediate, or egress with respect to aparticular LSP.

[0281] The function of each of the modules was described in the previoussection. To complement those descriptions, the role of each of themodules in each of these nodes is described herein with respect to theexemplary establishment of a new service. For clarity, this example willparallel the example provided earlier of the addition of LSP-11(working) and LSP-12 (protection) to SMDN 100 of FIG. 1.

[0282] To begin, a service request is received at node N4 of FIG. 1 forthe establishment of an LSP between N4 and N2 of SMDN 100. Assume that,at the arrival of the request, the network was in a state captured byFIG. 1 and TABLE 1 of FIG. 2. Referring to FIG. 18, the service requestis received and serviced by service agent (SA) 1802 of ingress node 1800via either user network interface (UNI) or simple network managementprotocol (SNMP) signaling. Details on SNMP and ATM UNI can be found inRFC2570 and The ATM Forum, “ATM User Network Interface (UNI) version4.1,” November 2002 (herein “UNI4.1”), each incorporated herein byreference in its entirety. The SA provides a translation between theapplication-layer request for the service and path management (PM)function 1804, which is responsible for the establishment and monitoringof the health of the LSP and its protection path. As illustrated, the PMincludes glue logic 1806 and standard and extended signaling components1808. Glue logic 1806 allows the PM to interface with various othermodules within the node that it needs to communicate with to accomplishits task, while signaling sub-module 1808 allows the PM to communicateinformation about the new paths to other nodes in the SMDN.

[0283] Specifically, glue logic 1806 allows the PM to communicate withpath computation (PC) module 1812 to determine two suitable disjoint (ifpossible) paths for working and protection LSPs. The PC in turninterfaces to network topology (NT) module 1818 to gather currenttopology information (e.g., via link-state architecture (LSA) messages)used in the path computation. In this example, it can be assumed that PC1812 returned the disjoint paths (N4-L3-N5-L4-N2) and (N4-L1-N1-L2-N2)to PM (possibly among other disjoint pairs) to consider as working andprotection paths for the newly requested service.

[0284] Glue logic 1806 also allows the PM to communicate withbandwidth-sharing information (BSI) module 1810. The BSI module isresponsible for managing the sharing of information related toprotection in the SMDN associated with LPS-11 and LSP-12. Its functionin N4 is to keep track of the sharing status at each of the incidentlinks to N4 (namely L1 and L3). Thus, it does the bookkeeping forsharing for N4 and thus maintains the information corresponding to rowsL1 and L3 of TABLE 1 (and, after update, TABLE 2 of FIG. 3). The BSI isalso responsible for providing the PM with working-path information thatit needs to share (along the protection path of the new LSP) to keep thenetwork up-to-date with the sharing information.

[0285] The PM also interfaces via glue logic 1806 to protectionswitching (PS) module 1814 to indicate by which path (working orprotection) incoming traffic will exit N4. As discussed before, this maybe implemented by activating one or another of the two NHLFE entries inthe (FTN) map (represented here by PS sub-module NHLFE 1816). During thecreation of a service, the traffic is mapped to the NHLFE entrycorresponding to the working LSP. The entry selection decision (andconsequently the choice of working or protection path) may be overriddenby information (e.g., via BDI-OAM packets) received via OAM module 1820indicating a failure somewhere along the downstream working path, inwhich case the traffic mapping is switched to the other NHLFE entrycorresponding to the protection LSP. Note that this switching of trafficbetween working and protection can also be forced via a networkmanagement command. OAM 1820 also functions to insert OAM packets (e.g.,FFD-OAM and FDI-OAM) into the working and protection paths to supportmonitoring and failure detections in the SMDN.

[0286] Referring next to FIG. 19, labels 1902 and 1904 refer tofunctional modules that are active in an intermediate node along theworking path and the protection path, respectively, of the SMDN. Withrespect to the present example, module 1902 depicts the operation of anode (e.g., N5) that is along working path LSP-11, and module 1904depicts the operation of a node (e.g., N1) that is along protection pathLSP-12.

[0287] As illustrated by module 1902, the PM in a working pathintermediate node relays standard signaling along the path and directsincoming label-switched traffic to the next hop along the working pathas indicated by the active entry (NHLFE) in the local incoming label map(ILM). If necessary (e.g., if a failure is detected), the local OAMfunction may insert FDI-OAM packets into relevant LSPs to supportmonitoring and failure detection in the SMDN.

[0288] As illustrated by module 1904, the PM in a protection pathintermediate node performs similarly to the PM in the working pathintermediate node but has some additional active modules as well.Namely, extended-signaling functions within the PM allow for the receiptof sharing information about the working path to be communicated via theglue logic to the local BSI module. As in the ingress node, the BSI willmaintain and update sharing information associated with the node'sincident links. For example, if module 1904 represents the activemodules in intermediate node N1 along LSP-12, then it will store thesharing information corresponding to its incident links L1 and L2,namely rows L1 and L2 of TABLE 1 (and, after the establishment of thenew LSP, rows L1 and L2 of TABLE 2).

[0289] Finally, in an egress node (illustrated by exemplary node 2000 ofFIG. 20), the PM serves as a terminal point in the LSP signaling chainfor both standard and extended signaling. It also interfaces to the PSto configure the two NHLFE entries corresponding to working andprotection LSPs. The incoming label selects the NHLFE entry that, asdiscussed before, dictates the next action on the packet. In this case,NHLFE traffic is extracted (corresponding to which path is activelycarrying traffic) and passed on to the node-attached client packet. Inthe case of penultimate hop popping, there might not be an NHLFE at theegress node. In this case, the previous node pops the label and the nextoperation is implicitly derived from the packet itself (for example,Internet protocol (IP) will be used to route the IP packet). An OAMmodule in the egress node extracts OAM packets from each LSP and insertsthem in the corresponding LSPs in the reverse direction to supportmonitoring and failure detection and recovery in the SMDN.

[0290] While the embodiments of this invention have been discussed withrespect to protection, they may equally well be applied to restoration,with the assumption that some or all of the calculations associated withpaths are calculated after the detection of a failure.

[0291] While this invention has been described with respect toprotection associated with situations where the protection was directedto single-point failures, the concepts, and, in particular, thelink-state description, may be extended to multiple-point failuresituations.

[0292] While various calculations within this invention have beendescribed with respect to unit bandwidth services, these calculationscould equally well be applied to non-unit bandwidth services, (e.g., byequating non-unit bandwidth services to an equivalent number ofunit-bandwidth services) as would be understood to one skilled in theart.

[0293] While this invention has been described with respect todissemination of sharing information by specific mechanisms includingOSPF, alternative means for dissemination of information within meshnetworks could also be used (e.g., RSVP-TE, TCP/IP connections,constrained routing label distribution protocol (CR-LDP)), and SNMP) aswould be understood to one skilled in the art.

[0294] While aspects of this invention has been described in one or moreembodiments in the context of mesh data networks running one or more ofthe multiprotocol-label switched (MPLS) traffic engineering extensionprotocols, those aspects are equally applicable to a wide class ofnetworks where traffic engineering is supported, these networksclassified as providing virtual-circuit services, as would be understoodto one skilled in the art.

[0295] Note that, in one or more embodiments of the present invention, anode is assumed to have the intelligence for computations associatedwith aspects of the invention (e.g., for determining sharing, andcalculating and reserving restoration bandwidth on links), however, aswould be understood to one skilled in the art, a regional manager thatsupports one or more nodes can be substituted for the node'sintelligence while remaining within the spirit and intent of the presentinvention. In this document, it should be understood that, in somecases, the regional manager may manage only a single node and be hostedby that node.

[0296] While this invention has been described with reference toillustrative embodiments, this description should not be construed in alimiting sense. Various modifications of the described embodiments, aswell as other embodiments of the invention, which are apparent topersons skilled in the art to which the invention pertains are deemed tolie within the principle and scope of the invention as expressed in thefollowing claims.

[0297] Although the steps in the following method claims, if any, arerecited in a particular sequence with corresponding labeling, unless theclaim recitations otherwise imply a particular sequence for implementingsome or all of those steps, those steps are not necessarily intended tobe limited to being implemented in that particular sequence.

What is claimed is:
 1. A method comprising: representing, in a networkdata structure, information associated with a mesh network having aplurality of nodes interconnected by a plurality of links, wherein thenetwork data structure comprises, for each link in the network and eachnode or other link in the network, a representation of a minimum amountof protection bandwidth required to be reserved on said each link torestore service upon failure of said node or other link; receiving arequest for a new service in the network, wherein the new service isrepresented by a service data structure comprising an identification ofeach link and transit node in a primary path for the new service;determining, using the network and service data structures, whether thenew service requires additional protection bandwidth to be reserved onany link in the network; and updating the network data structure if anyadditional protection bandwidth is determined to be required for the newservice.
 2. The invention of claim 1, wherein the service data structurefurther comprises an identification of bandwidth associated with the newservice.
 3. The invention of claim 1, wherein the network is avirtual-circuit mesh data network that transmits packetized data.
 4. Theinvention of claim 1, wherein the network data structure is distributedover the network such that at least one node in the network does nothave all of the information in the network data structure.
 5. Theinvention of claim 1, wherein each of the nodes in the network has allof the information in the network data structure.
 6. The invention ofclaim 1, further comprising, in response to the new service request,determining a restoration path for the new service in the network usingthe network data structure.
 7. The invention of claim 1, wherein: thenetwork data structure is an array of vectors, wherein: each vector inthe array corresponds to a different link in the network; each vector inthe array has a plurality of entries corresponding to the nodes andlinks in the network; for a first vector corresponding to a first link,each entry in the first vector corresponding to a node or other linkidentifies the minimum amount of protection bandwidth required to bereserved on the first link to restore service upon failure of the nodeor other link; and the service data structure is a primary path vectorhaving a plurality of entries corresponding to the nodes and links inthe network, wherein: each entry of the primary path vector identifieswhether the corresponding node or link is part of the primary path forthe new service.
 8. The invention of claim 7, wherein determiningwhether the new service requires any additional protection bandwidth tobe reserved on a link A in the network comprises applying a vectoraddition operation between the primary path vector corresponding to thenew service request and the vector of the array corresponding to thelink A to form a result vector, and comparing the maximum value in theresult vector with the bandwidth already reserved on the link A todetermine whether any additional protection bandwidth is required forthe new service.
 9. The invention of claim 8, wherein the additionalprotection bandwidth is required and is reserved if any result vectorentry is greater than the bandwidth already reserved on the link. 10.The invention of claim 8, wherein the vector addition operation isapplied between the primary path vector and each vector in the arraycorresponding to each different link in a restoration path for the newservice.
 11. The invention of claim 1, wherein the service datastructure is primary path node-link vector V_(pnl).
 12. The invention ofclaim 1, wherein a compact version of the network data structure is usedto reduce the amount of data that needs to be transmitted in the networkto disseminate the information about each link.
 13. The invention ofclaim 12, wherein transmission control protocol/Internet protocol(TCP/IP) connections are used for the dissemination.
 14. The inventionof claim 12, wherein the compact representation is a node aggregatevector V_(na) wherein each element of V_(na) corresponds to a node inthe network wherein the element's value is a function of the maximum ofreservation bandwidths reserved on all links incident to the node. 15.The invention of claim 12, wherein the dissemination is accomplishedusing a link-state routing protocol.
 16. The invention of claim 1,wherein an incremental version of the network data structure is used toreduce the amount of data that is transmitted in the network todisseminate the information.
 17. A network manager for a mesh networkhaving a plurality of nodes interconnected by a plurality of links, thenetwork manager adapted to: represent, in a network data structure,information associated with the mesh network, wherein the network datastructure comprises, for each link in the network and each node or otherlink in the network, a representation of a minimum amount of protectionbandwidth required to be reserved on said each link to restore serviceupon failure of said node or other link; receive a request for a newservice in the network, wherein the new service is represented by aservice data structure comprising an identification of each link andtransit node in a primary path for the new service; determine, using thenetwork and service data structures, whether the new service requiresadditional protection bandwidth to be reserved on any link in thenetwork; and update the network data structure if any additionalprotection bandwidth is determined to be required for the new service.18. The invention of claim 17, wherein the network manager isdistributed over the network.
 19. The invention of claim 17, wherein thenetwork manager is located at a single node of the network.
 20. Theinvention of claim 17, wherein a compact version of the network datastructure is used to reduce the amount of data that needs to betransmitted in the network to disseminate the information about eachlink.